9:07 a.m.
[
https://issues.jboss.org/browse/WFLY-5480?page=com.atlassian.jira.plugin....
]
Flavia Rainone commented on WFLY-5480:
--------------------------------------
I found out the following: the bug is caused by a race condition, probably at
org.apache.activemq.artemis.core.message.impl.MessageImpl.
If I put a breakpoint at the line that reads the problematic index (the line that reads
the value of 2130706436), the index value is always read correctly with the breakpoint on
(the read number varies, but is usually approximately 2280). Would the bug be on the
server side, the client would always read an incorrect number no matter if there is a
breakpoint or not.
Reading the code of MessageImpl, I saw that there is already some code synchronizing the
access to the buffer to handle race conditions involving the problematic buffer... now I
wonder if this bug is related.
At the MessagingImpl constructor:
// This MUST be synchronized using the monitor on the other message to prevent it
running concurrently
// with getEncodedBuffer(), otherwise can introduce race condition when delivering
concurrently to
// many subscriptions and bridging to other nodes in a cluster
synchronized (other) {
In case somebody is wondering what is end of body position about (the value that is
sometimes read as being 2130706436), we can see a description at the line that encodes
it:
// Clebert: I've started sending this on encoding due to conversions between
protocols
// and making sure we are not losing the buffer start position between
protocols
this.endOfBodyPosition = bodySize;
A more deep investigation on the side of amq code will be need to find the cause of all
this (I'll need to edit the code to dig deeper).
Btw, I am using wildfly master to run the tests.
SPEC JMS 2007 benchmark fails with AMQ214013: Failed to decode
packet
---------------------------------------------------------------------
Key: WFLY-5480
URL: https://issues.jboss.org/browse/WFLY-5480
Project: WildFly
Issue Type: Bug
Components: JMS
Reporter: Ondřej Kalman
Assignee: Flavia Rainone
Priority: Blocker
But I have another problem, when I removed trace logs from config I'm not able to run
benchmark on localhost to the end, because clients starts getting :
AMQ214013: Failed to decode packet
java.lang.IndexOutOfBoundsException: readerIndex: 2130706436 (expected: 0 <=
readerIndex <= writerIndex(2943))
at io.netty.buffer.AbstractByteBuf.readerIndex(AbstractByteBuf.java:73)
at io.netty.buffer.WrappedByteBuf.readerIndex(WrappedByteBuf.java:99)
at
org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readerIndex(ChannelBufferWrapper.java:405)
at
org.apache.activemq.artemis.core.message.impl.MessageImpl.decode(MessageImpl.java:1052)
at
org.apache.activemq.artemis.core.message.impl.MessageImpl.decodeFromBuffer(MessageImpl.java:459)
at
org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionReceiveMessage.decode(SessionReceiveMessage.java:94)
at
org.apache.activemq.artemis.core.protocol.ClientPacketDecoder.decode(ClientPacketDecoder.java:42)
at
org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:371)
at
org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1374)
at
org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteCha
nnel.java:131)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)
and
Thread-9 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
java.lang.NumberFormatException: null
at java.lang.Integer.parseInt(Integer.java:542)
at java.lang.Integer.valueOf(Integer.java:766)
at
org.apache.activemq.artemis.utils.TypedProperties.getIntProperty(TypedProperties.java:280)
at
org.apache.activemq.artemis.core.message.impl.MessageImpl.getIntProperty(MessageImpl.java:811)
at
org.apache.activemq.artemis.jms.client.ActiveMQMessage.getIntProperty(ActiveMQMessage.java:578)
at org.spec.jms.agents.SPECWorkerThread.receivedMessage(SPECWorkerThread.java:849)
at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:820)
at
org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
at
org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
at
org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
at
org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
at
org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Thread-3 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
javax.jms.IllegalStateException: AMQ119027: Could not find reference on consumer ID=0,
messageId = 104,833 queue = 127\.0\.0\.1_VM1_SPAgent7_0.SP_CallForOffersEH_7_EHID_1PF0
at
org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:410)
at
org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendACK(ActiveMQSessionContext.java:461)
at
org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.acknowledge(ClientSessionImpl.java:765)
at
org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.doAck(ClientConsumerImpl.java:1212)
at
org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.flushAcks(ClientConsumerImpl.java:830)
at
org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.flushAcks(ClientSessionImpl.java:1852)
at
org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:501)
at
org.apache.activemq.artemis.core.client.impl.DelegatingSession.commit(DelegatingSession.java:159)
at
org.apache.activemq.artemis.jms.client.ActiveMQSession.commit(ActiveMQSession.java:218)
at
org.spec.jms.eventhandler.sp.SP_CallForOffersEH.handleMessage(SP_CallForOffersEH.java:306)
at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:821)
at
org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
at
org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
at
org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
at
org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
at
org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecut
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
3935
days inactive
3935
days old
0 comments
1 participants
participants (1)
-
Flavia Rainone (JIRA)