[JBoss JIRA] (HRJS-24) "Unknown version:-96" appears sometimes on the server while put/get operation
7:52 a.m.
[
https://issues.jboss.org/browse/HRJS-24?page=com.atlassian.jira.plugin.sy...
]
Galder Zamarreño commented on HRJS-24:
--------------------------------------
This sounds like a bug in the Hot Rod server implementation:
I've added some extra logic to the Hot Rod server to dump the buffer when its get
errors like this and I see this:
{code}
2017-01-13 13:38:26,250 TRACE [org.infinispan.server.hotrod.HotRodDecoder]
(HotRodServerWorker-7-1) Decode using instance @2771220b
2017-01-13 13:38:26,250 TRACE [org.infinispan.server.hotrod.HotRodDecoder]
(HotRodServerWorker-7-1) Invalid magic id error encountered, the buffer contains:
0300033130307703313030A06A190100000300033130317703313031A06B190100000300033130327703313032A06C190100000300033130337703313033A06D190100000300033130347703313034A06E190100000300033130357703313035A06F190100000300033130367703313036A070190100000300033130377703313037A071190100000300033130387703313038A072190100000300033130397703313039A0731901
2017-01-13 13:38:26,255 ERROR [org.infinispan.server.hotrod.CacheDecodeContext]
(HotRodServerWorker-7-1) ISPN005003: Exception reported:
org.infinispan.server.hotrod.InvalidMagicIdException: Error reading magic byte or message
id: 0
{code}
After analysing the remaining buffer, I see:
{code}
=> Analysis
?? ?? ?? ???? ?? 0300 033130307703313030
A0 6A 19 0100 00 0300 033130317703313031
A0 6B 19 0100 00 0300 033130327703313032
A0 6C 19 0100 00 0300 033130337703313033
...
{code}
So, most likely what happened is this: The server read fine up to operation and length of
cache name (0100), then it couldn't read flag because it didn't have enough bytes,
so it had to get another decode callback. When the decode callback comes in, it reads the
flag when it expects a new message.
The bug might be that before it reads the flag, it needs to check whether there are enough
bytes to read the flag. This check is not currently being done. I need to check how the
new decoders work WRT tracking down what has been read...etc and check with Will.
"Unknown version:-96" appears sometimes on the server while
put/get operation
-----------------------------------------------------------------------------
Key: HRJS-24
URL: https://issues.jboss.org/browse/HRJS-24
Project: Infinispan Javascript client
Issue Type: Bug
Affects Versions: 0.3.0
Reporter: Anna Manukyan
Assignee: Galder Zamarreño
I have extended the radargun with nodejs plugin so that later it is possible to test the
xsite failover.
The issue is that sometimes under the load while performing a put/get request over nodejs
client, I am getting the following exception on the server:
{code}
14:02:56,885 ERROR [org.infinispan.server.hotrod.CacheDecodeContext]
(HotRodServerWorker-7-1) ISPN005003: Exception reported:
org.infinispan.server.hotrod.UnknownVersionException: Unknown version:-96
at org.infinispan.server.hotrod.HotRodDecoder.readHeader(HotRodDecoder.java:199)
at org.infinispan.server.hotrod.HotRodDecoder.decodeHeader(HotRodDecoder.java:129)
at org.infinispan.server.hotrod.HotRodDecoder.decode(HotRodDecoder.java:87)
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:387)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:245)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:154)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:354)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:145)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:28)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:154)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:354)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:145)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:1078)
at
io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:872)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:359)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:275)
at
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)
at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)
at java.lang.Thread.run(Thread.java:745)
14:02:56,888 ERROR [org.infinispan.server.hotrod.HotRodEncoder] (HotRodServerWorker-7-1)
ISPN005023: Exception encoding message ErrorResponse{version=-96,
messageId=-9223372036854775808, operation=ErrorResponse, status=UnknownVersion,
msg=org.infinispan.server.hotrod.UnknownVersionException: Unknown version:-96}:
scala.MatchError: -96 (of class java.lang.Byte)
at org.infinispan.server.hotrod.HotRodEncoder.getEncoder(HotRodEncoder.scala:79)
at org.infinispan.server.hotrod.HotRodEncoder.encode(HotRodEncoder.scala:38)
at io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:107)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeWriteNow(ChannelHandlerInvokerUtil.java:157)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeWrite(DefaultChannelHandlerInvoker.java:372)
at
io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:400)
at
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:266)
at
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:272)
at
io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1186)
at io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:286)
at
org.infinispan.server.hotrod.HotRodExceptionHandler.exceptionCaught(HotRodExceptionHandler.java:31)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeExceptionCaughtNow(ChannelHandlerInvokerUtil.java:64)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeExceptionCaught(DefaultChannelHandlerInvoker.java:111)
at
io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at
io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:131)
at
io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeExceptionCaughtNow(ChannelHandlerInvokerUtil.java:64)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeExceptionCaught(DefaultChannelHandlerInvoker.java:111)
at
io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at
io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:131)
at
io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeExceptionCaughtNow(ChannelHandlerInvokerUtil.java:64)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeExceptionCaught(DefaultChannelHandlerInvoker.java:111)
at
io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at
io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:131)
at
io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:78)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeExceptionCaughtNow(ChannelHandlerInvokerUtil.java:64)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeExceptionCaught(DefaultChannelHandlerInvoker.java:111)
at
io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at
io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:131)
at
io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeExceptionCaughtNow(ChannelHandlerInvokerUtil.java:64)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeExceptionCaught(DefaultChannelHandlerInvoker.java:111)
at
io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at
io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:131)
at
io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeExceptionCaughtNow(ChannelHandlerInvokerUtil.java:64)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeExceptionCaught(DefaultChannelHandlerInvoker.java:111)
at
io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at
io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:131)
at
io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:1066)
at org.infinispan.server.hotrod.HotRodDecoder.decode(HotRodDecoder.java:124)
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:387)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:245)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:154)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:354)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:145)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:28)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:154)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:354)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:145)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:1078)
at
io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:872)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:359)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:275)
at
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)
at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)
at java.lang.Thread.run(Thread.java:745)
{code}
I am using the infinispan-js-client 0.3.0. For server different versions are tried -
8.4.2.Final, 9.0.0.Alpha4 - the issue still appears. For 20000 requests I am getting this
error for about 3-4 times (maybe more, because after the error appears I am getting an
exception on Java side).
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
2916
days inactive
2916
days old
infinispan-issues@lists.jboss.org
0 comments
1 participants
participants (1)
-
Galder Zamarreño (JIRA)