[JBoss JIRA] (ISPN-4717) Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-4717?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-4717:
-----------------------------------
Description:
The way Hot Rod protocol deals with suspected exceptions is hacky. It inspects the error message to detect whether a SuspectException has been passed in. Instead, suspect exceptions should have a dedicated error code so that clients can handle appropriately.
On top of that, another exception that should be handled more silently and failover is when a cache is stopping or is stopped. -Currently, this produces the following log messages without affecting functionality- Scrap that, it does get propagated to the client without being able to failover, so it's a bug:
{code}2014-09-11 08:11:04,984 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005003: Exception reported
java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:744)
2014-09-11 08:11:04,990 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005009: Unexpected error before any request parameters read
io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:417)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.server.hotrod.HotRodDecoder.createServerException(HotRodDecoder.scala:204)
at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:77)
at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
... 12 more
Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
... 14 more
2014-09-11 08:11:04,991 WARN [Codec20] (ForkThread-1,DistTopologyChangeUnderLoadTest) ISPN004005: Error received from the server: io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
{code}
Cache stopping/stopped should have a different error code too so that clients can handle it properly.
was:
The way Hot Rod protocol deals with suspected exceptions is hacky. It inspects the error message to detect whether a SuspectException has been passed in. Instead, suspect exceptions should have a dedicated error code so that clients can handle appropriately.
On top of that, another exception that should be handled more silently and failover is when a cache is stopping or is stopped. Currently, this produces the following log messages without affecting functionality:
{code}2014-09-11 08:11:04,984 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005003: Exception reported
java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:744)
2014-09-11 08:11:04,990 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005009: Unexpected error before any request parameters read
io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:417)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.server.hotrod.HotRodDecoder.createServerException(HotRodDecoder.scala:204)
at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:77)
at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
... 12 more
Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
... 14 more
2014-09-11 08:11:04,991 WARN [Codec20] (ForkThread-1,DistTopologyChangeUnderLoadTest) ISPN004005: Error received from the server: io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
{code}
Cache stopping/stopped should have a different error code too so that clients can handle it properly.
> Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
> ----------------------------------------------------------------------------------
>
> Key: ISPN-4717
> URL: https://issues.jboss.org/browse/ISPN-4717
> Project: Infinispan
> Issue Type: Enhancement
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Fix For: 7.0.0.Beta2
>
>
> The way Hot Rod protocol deals with suspected exceptions is hacky. It inspects the error message to detect whether a SuspectException has been passed in. Instead, suspect exceptions should have a dedicated error code so that clients can handle appropriately.
> On top of that, another exception that should be handled more silently and failover is when a cache is stopping or is stopped. -Currently, this produces the following log messages without affecting functionality- Scrap that, it does get propagated to the client without being able to failover, so it's a bug:
> {code}2014-09-11 08:11:04,984 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005003: Exception reported
> java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
> at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
> at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
> at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at java.lang.Thread.run(Thread.java:744)
> 2014-09-11 08:11:04,990 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005009: Unexpected error before any request parameters read
> io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:417)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
> at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.server.hotrod.HotRodDecoder.createServerException(HotRodDecoder.scala:204)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:77)
> at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
> at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
> ... 12 more
> Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
> at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
> at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
> at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
> at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
> at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
> at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
> at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
> at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
> at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
> at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
> ... 14 more
> 2014-09-11 08:11:04,991 WARN [Codec20] (ForkThread-1,DistTopologyChangeUnderLoadTest) ISPN004005: Error received from the server: io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
> {code}
> Cache stopping/stopped should have a different error code too so that clients can handle it properly.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4029) NotifyingFutureTest.testExceptionOtherThread1 && testDoneOtherThread2 random failures
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-4029?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec updated ISPN-4029:
--------------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2859
> NotifyingFutureTest.testExceptionOtherThread1 && testDoneOtherThread2 random failures
> -------------------------------------------------------------------------------------
>
> Key: ISPN-4029
> URL: https://issues.jboss.org/browse/ISPN-4029
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 6.0.1.Final
> Reporter: Dan Berindei
> Assignee: Sebastian Łaskawiec
> Labels: testsuite_stability
> Fix For: 7.0.0.Final
>
>
> {noformat}
> 16:58:33,402 ERROR (testng-NotifyingFutureTest:) [UnitTestTestNGListener] Test testExceptionOtherThread1(org.infinispan.commons.util.concurrent.NotifyingFutureTest) failed.
> java.lang.AssertionError: expected [true] but found [false]
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertTrue(Assert.java:42)
> at org.testng.Assert.assertTrue(Assert.java:52)
> at org.infinispan.commons.util.concurrent.NotifyingFutureTest.testException(NotifyingFutureTest.java:150)
> at org.infinispan.commons.util.concurrent.NotifyingFutureTest.testExceptionOtherThread(NotifyingFutureTest.java:67)
> at org.infinispan.commons.util.concurrent.NotifyingFutureTest.testExceptionOtherThread1(NotifyingFutureTest.java:46)
> {noformat}
> The test submits a task to a ThreadPoolExecutor that notifies a NotifyingFutureImpl and assumes that when the NotifyingFutureImpl is done, the original Future is also done. That's clearly not happening some of the time.
> * Jenkins
> ** RHEL, Oracle JDK7
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/FUNC/job/e...
> ** RHEL, OpenJDK7
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/FUNC/job/e...
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4029) NotifyingFutureTest.testExceptionOtherThread1 && testDoneOtherThread2 random failures
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-4029?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec reassigned ISPN-4029:
-----------------------------------------
Assignee: Sebastian Łaskawiec (was: Dan Berindei)
> NotifyingFutureTest.testExceptionOtherThread1 && testDoneOtherThread2 random failures
> -------------------------------------------------------------------------------------
>
> Key: ISPN-4029
> URL: https://issues.jboss.org/browse/ISPN-4029
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 6.0.1.Final
> Reporter: Dan Berindei
> Assignee: Sebastian Łaskawiec
> Labels: testsuite_stability
> Fix For: 7.0.0.Final
>
>
> {noformat}
> 16:58:33,402 ERROR (testng-NotifyingFutureTest:) [UnitTestTestNGListener] Test testExceptionOtherThread1(org.infinispan.commons.util.concurrent.NotifyingFutureTest) failed.
> java.lang.AssertionError: expected [true] but found [false]
> at org.testng.Assert.fail(Assert.java:94)
> at org.testng.Assert.failNotEquals(Assert.java:494)
> at org.testng.Assert.assertTrue(Assert.java:42)
> at org.testng.Assert.assertTrue(Assert.java:52)
> at org.infinispan.commons.util.concurrent.NotifyingFutureTest.testException(NotifyingFutureTest.java:150)
> at org.infinispan.commons.util.concurrent.NotifyingFutureTest.testExceptionOtherThread(NotifyingFutureTest.java:67)
> at org.infinispan.commons.util.concurrent.NotifyingFutureTest.testExceptionOtherThread1(NotifyingFutureTest.java:46)
> {noformat}
> The test submits a task to a ThreadPoolExecutor that notifies a NotifyingFutureImpl and assumes that when the NotifyingFutureImpl is done, the original Future is also done. That's clearly not happening some of the time.
> * Jenkins
> ** RHEL, Oracle JDK7
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/FUNC/job/e...
> ** RHEL, OpenJDK7
> https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/FUNC/job/e...
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4717) Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
by Galder Zamarreño (JIRA)
Galder Zamarreño created ISPN-4717:
--------------------------------------
Summary: Hot Rod 2.0 should add error codes for suspected nodes and stopping/stopped caches
Key: ISPN-4717
URL: https://issues.jboss.org/browse/ISPN-4717
Project: Infinispan
Issue Type: Enhancement
Reporter: Galder Zamarreño
Assignee: Galder Zamarreño
Fix For: 7.0.0.Beta2
The way Hot Rod protocol deals with suspected exceptions is hacky. It inspects the error message to detect whether a SuspectException has been passed in. Instead, suspect exceptions should have a dedicated error code so that clients can handle appropriately.
On top of that, another exception that should be handled more silently and failover is when a cache is stopping or is stopped. Currently, this produces the following log messages without affecting functionality:
{code}2014-09-11 08:11:04,984 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005003: Exception reported
java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:744)
2014-09-11 08:11:04,990 ERROR [HotRodDecoder] (HotRodServerWorker-6-1) ISPN005009: Unexpected error before any request parameters read
io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:417)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
at org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:471)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.server.hotrod.HotRodDecoder.createServerException(HotRodDecoder.scala:204)
at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:77)
at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:61)
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:362)
... 12 more
Caused by: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:94)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:33)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1490)
at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:968)
at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:960)
at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:485)
at org.infinispan.server.core.AbstractProtocolDecoder.put(AbstractProtocolDecoder.scala:252)
at org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeValue(AbstractProtocolDecoder.scala:207)
at org.infinispan.server.core.AbstractProtocolDecoder.decodeDispatch(AbstractProtocolDecoder.scala:73)
... 14 more
2014-09-11 08:11:04,991 WARN [Codec20] (ForkThread-1,DistTopologyChangeUnderLoadTest) ISPN004005: Error received from the server: io.netty.handler.codec.DecoderException: org.infinispan.server.hotrod.HotRodException: java.lang.IllegalStateException: Default cache is in 'STOPPING' state and this is an invocation not belonging to an on-going transaction, so it does not accept new invocations. Either restart it or recreate the cache container.
{code}
Cache stopping/stopped should have a different error code too so that clients can handle it properly.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4406) Stabilise security manager integration testsuite
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4406?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4406:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1091373|https://bugzilla.redhat.com/show_bug.cgi?id=1091373] from MODIFIED to ON_QA
> Stabilise security manager integration testsuite
> ------------------------------------------------
>
> Key: ISPN-4406
> URL: https://issues.jboss.org/browse/ISPN-4406
> Project: Infinispan
> Issue Type: Bug
> Components: Security
> Reporter: Galder Zamarreño
> Assignee: Galder Zamarreño
> Fix For: 7.0.0.Alpha5
>
>
> integrationtests/security-manager-it tests randomly fail, which might be related to the fact that a lot of these tests rely on static initialisation, and I think that's causing messing up between tests initialising/uninitialising these static variables. The fact that the errors are random seems to back this up.
> However, enabling sequential testing results in:
> {code}[ERROR] java.lang.SecurityException: ISPN000287: Unauthorized access: subject 'null' lacks 'LIFECYCLE' permission
> [ERROR] at org.infinispan.security.impl.AuthorizationHelper.checkPermission(AuthorizationHelper.java:56)
> [ERROR] at org.infinispan.security.impl.AuthorizationHelper.checkPermission(AuthorizationHelper.java:70)
> [ERROR] at org.infinispan.security.impl.AuthorizationHelper.checkPermission(AuthorizationHelper.java:75)
> [ERROR] at org.infinispan.manager.DefaultCacheManager.getStatus(DefaultCacheManager.java:677)
> [ERROR] at org.infinispan.test.fwk.TestCacheManagerFactory$PerThreadCacheManagers.checkManagersClosed(TestCacheManagerFactory.java:435)
> [ERROR] at org.infinispan.test.fwk.TestCacheManagerFactory.testFinished(TestCacheManagerFactory.java:422)
> [ERROR] at org.infinispan.test.fwk.UnitTestTestNGListener.onFinish(UnitTestTestNGListener.java:104){code}
> This error is also present in paralell testing but in this case it's just logged rather than halting the process.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4237) Security manager test QueryAuthorizationTest.testQuery fails
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4237?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4237:
-----------------------------------------------
Dave Stahl <dstahl(a)redhat.com> changed the Status of [bug 1091373|https://bugzilla.redhat.com/show_bug.cgi?id=1091373] from MODIFIED to ON_QA
> Security manager test QueryAuthorizationTest.testQuery fails
> ------------------------------------------------------------
>
> Key: ISPN-4237
> URL: https://issues.jboss.org/browse/ISPN-4237
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 7.0.0.Alpha3
> Reporter: Vojtech Juranek
> Assignee: Tristan Tarrant
> Labels: testsuite_stability
> Fix For: 7.0.0.Beta2
>
> Attachments: QueryAuthorizationTest.log.zip
>
>
> {{org.infinispan.security.QueryAuthorizationTest.testQuery}} fails (no matter which JDK is used) with
> {noformat}
> java.lang.IllegalArgumentException: Indexing was not enabled on this cache. interface org.hibernate.search.spi.SearchFactoryIntegrator not found in registry
> at org.infinispan.query.impl.ComponentRegistryUtils.getComponent(ComponentRegistryUtils.java:27)
> at org.infinispan.query.impl.ComponentRegistryUtils.getComponent(ComponentRegistryUtils.java:20)
> at org.infinispan.query.impl.SearchManagerImpl.<init>(SearchManagerImpl.java:42)
> at org.infinispan.query.Search.getSearchManager(Search.java:21)
> at org.infinispan.security.QueryAuthorizationTest.queryTest(QueryAuthorizationTest.java:97)
> at org.infinispan.security.QueryAuthorizationTest.access$300(QueryAuthorizationTest.java:32)
> at org.infinispan.security.QueryAuthorizationTest$4.run(QueryAuthorizationTest.java:113)
> at org.infinispan.security.QueryAuthorizationTest$4.run(QueryAuthorizationTest.java:109)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.infinispan.security.QueryAuthorizationTest.testQuery(QueryAuthorizationTest.java:109)
> {noformat}
> Example of test failure in Jenkins on RHEL6 and Oracle JDK 7 is e.g. [here|https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-tes...].
> Maybe it's some setup issue, as on Jenkins it fails on all platforms, while I;m not able to reproduce it on my machine.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months