[JBoss JIRA] (WFLY-5822) Clustering performance regression in ejbremote-dist-sync scenario
by Richard Achmatowicz (JIRA)
[ https://issues.jboss.org/browse/WFLY-5822?page=com.atlassian.jira.plugin.... ]
Richard Achmatowicz edited comment on WFLY-5822 at 1/11/16 8:18 PM:
--------------------------------------------------------------------
With regard to remote accesses of keys:
7.0.0 jobs:
- run with the Byteman rule RetrieveFromRemoteSource which checks for invocations of the method BaseDistributionInterceptor.retrieveFromRemoteSource
- output found for this rule in the job output for random cluster nodes (in other words, on one run, perf18 has to remotely retrieve a key, on another run, perf21 has to retrieve a remote key)
- because this is a stress test and failures are not triggered, unless there is an ISPN rebalance for reasons other than failure, the node with the remote key will have to make that extra invocation on every access to that key
6.4.0 jobs:
- run with the Byteman rule RetrieveFromRemoteSource which checks for invocations of the method BaseDistributionInterceptor.retrieveFromRemoteSource
- no output found for this rule in the job output for any cluster node
These "extra" remote accesses are triggered during the resetting of the weakAffinity, after a successful invocation and before the invocation result is returned to the client. The server will set the weak affinity of the session by calling MethodInvocationMessageHandler.getWeakAffinity(sessionID). This invocation passes in turn through DistributableCache.getWeakAffinity(..), InfinispanBeanManager.getWeakAffinity(...) until it looks up the primary owner of the session via InfinispanBeanManager.locatePrimaryOwner(sessionID) which returns a Node object identifying the Node in the Group which should be used for weak affinity.
However, the Node needs to be translated into an Address / string representation. So the code makes a call to the Registry (a distributed cache) to lookup the String name of the host. Unfortunately, there is no guarantee that the cache entry in the Registry for a Node perf18 is resident on perf18 - and so a call to retrieveFromRemote has to be made to get the cache entry from the other node(s).
I'm not certain that this is the cause of the performance problem, but it seems to be the reason why the extra remote call is made.
was (Author: rachmato):
With regard to remote accesses of keys:
7.0.0 jobs:
- run with the Byteman rule RetrieveFromRemoteSource which checks for invocations of the method BaseDistributionInterceptor.retrieveFromRemoteSource
- output found for this rule in the job output for random cluster nodes (in other words, on one run, perf18 has to remotely retrieve a key, on another run, perf21 has to retrieve a remote key)
- because this is a stress test and failures are not triggered, unless there is an ISPN rebalance for reasons other than failure, the node with the remote key will have to make that extra invocation on every access to that key
6.4.0 jobs:
- run with the Byteman rule RetrieveFromRemoteSource which checks for invocations of the method BaseDistributionInterceptor.retrieveFromRemoteSource
- no output found for this rule in the job output for any cluster node
These "extra" remote accesses are triggered during the resetting of the weakAffinity, after a successful invocation and before the invocation result is returned to the client. The server will set the weak affinity of the session by calling MethodInvocationMessageHandler.getWeakAffinity(sessionID). This invocation passes in turn through DistributableCache.getWeakAffinity(..), InfinispanBeanManager.getWeakAffinity(...) until it looks up the primary owner of the session via InfinispanBeanManager.locatePrimaryOwner(sessionID) which returns a Node object identifying the Node in the Group which should be used for weak affinity. However, the Node needs to be translated into an Address / string representation. So the code makes a call to the Registry (a distributed cache) to lookup the String name of the host. Unfortunately, there is no guarantee that the cache entry in the Registry for a Node perf18 is resident on perf18 - and so a call to retrieveFromRemote has to be made to get the cache entry from the other node(s).
I'm not certain that this is the cause of the performance problem, but it seems to be the reason why the extra remote call is made.
> Clustering performance regression in ejbremote-dist-sync scenario
> ------------------------------------------------------------------
>
> Key: WFLY-5822
> URL: https://issues.jboss.org/browse/WFLY-5822
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, EJB
> Affects Versions: 10.0.0.CR5
> Reporter: Michal Vinkler
> Assignee: Richard Achmatowicz
> Priority: Critical
>
> Compared to EAP 6, all SYNC scenarios have the same/better performance except of this one, wonder why?
> Compare these results:
> stress-ejbremote-dist-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
> ---------------------------------------
> Just for comparison: ejbremote REPL_SYNC scenario *performs well* on the other hand:
> stress-ejbremote-repl-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 3 months
[JBoss JIRA] (WFLY-5822) Clustering performance regression in ejbremote-dist-sync scenario
by Richard Achmatowicz (JIRA)
[ https://issues.jboss.org/browse/WFLY-5822?page=com.atlassian.jira.plugin.... ]
Richard Achmatowicz edited comment on WFLY-5822 at 1/11/16 8:16 PM:
--------------------------------------------------------------------
With regard to remote accesses of keys:
7.0.0 jobs:
- run with the Byteman rule RetrieveFromRemoteSource which checks for invocations of the method BaseDistributionInterceptor.retrieveFromRemoteSource
- output found for this rule in the job output for random cluster nodes (in other words, on one run, perf18 has to remotely retrieve a key, on another run, perf21 has to retrieve a remote key)
- because this is a stress test and failures are not triggered, unless there is an ISPN rebalance for reasons other than failure, the node with the remote key will have to make that extra invocation on every access to that key
6.4.0 jobs:
- run with the Byteman rule RetrieveFromRemoteSource which checks for invocations of the method BaseDistributionInterceptor.retrieveFromRemoteSource
- no output found for this rule in the job output for any cluster node
These "extra" remote accesses are triggered during the resetting of the weakAffinity, after a successful invocation and before the invocation result is returned to the client. The server will set the weak affinity of the session by calling MethodInvocationMessageHandler.getWeakAffinity(sessionID). This invocation passes in turn through DistributableCache.getWeakAffinity(..), InfinispanBeanManager.getWeakAffinity(...) until it looks up the primary owner of the session via InfinispanBeanManager.locatePrimaryOwner(sessionID) which returns a Node object identifying the Node in the Group which should be used for weak affinity. However, the Node needs to be translated into an Address / string representation. So the code makes a call to the Registry (a distributed cache) to lookup the String name of the host. Unfortunately, there is no guarantee that the cache entry in the Registry for a Node perf18 is resident on perf18 - and so a call to retrieveFromRemote has to be made to get the cache entry from the other node(s).
I'm not certain that this is the cause of the performance problem, but it seems to be the reason why the extra remote call is made.
was (Author: rachmato):
With regard to remote accesses of keys:
7.0.0 jobs:
- run with the Byteman rule RetrieveFromRemoteSource which checks for invocations of the method BaseDistributionInterceptor.retrieveFromRemoteSource
- output found for this rule in the job output for random cluster nodes (in other words, on one run, perf18 has to remotely retrieve a key, on another run, perf21 has to retrieve a remote key)
- because this is a stress test and failures are not triggered, unless there is an ISPN rebalance for reasons other than failure, the node with the remote key will have to make that extra invocation on every access to that key
6.4.0 jobs:
- run with the Byteman rule RetrieveFromRemoteSource which checks for invocations of the method BaseDistributionInterceptor.retrieveFromRemoteSource
- no output found for this rule in the job output for any cluster node
MOre investigation to follow.
> Clustering performance regression in ejbremote-dist-sync scenario
> ------------------------------------------------------------------
>
> Key: WFLY-5822
> URL: https://issues.jboss.org/browse/WFLY-5822
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, EJB
> Affects Versions: 10.0.0.CR5
> Reporter: Michal Vinkler
> Assignee: Richard Achmatowicz
> Priority: Critical
>
> Compared to EAP 6, all SYNC scenarios have the same/better performance except of this one, wonder why?
> Compare these results:
> stress-ejbremote-dist-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
> ---------------------------------------
> Just for comparison: ejbremote REPL_SYNC scenario *performs well* on the other hand:
> stress-ejbremote-repl-sync
> 7.0.0.ER2: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-str...]
> 6.4.0.GA: [throughput|http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-6x-str...]
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 3 months
[JBoss JIRA] (WFLY-5480) SPEC JMS 2007 benchmark fails with AMQ214013: Failed to decode packet
by Clebert Suconic (JIRA)
[ https://issues.jboss.org/browse/WFLY-5480?page=com.atlassian.jira.plugin.... ]
Clebert Suconic commented on WFLY-5480:
---------------------------------------
The Flow control thing I was mentioning was a feature we implemented on Artemis 1.2.0. it still not part of the Artemis fork for wildfly, regarding Network flow control. which is different from the credits from control implemented at artemis.
Now reading Flavia's comments it seems suspicious. can you try the latest Artemis (1.2.0) on Wildfly just for testing?
> SPEC JMS 2007 benchmark fails with AMQ214013: Failed to decode packet
> ---------------------------------------------------------------------
>
> Key: WFLY-5480
> URL: https://issues.jboss.org/browse/WFLY-5480
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Reporter: Ondřej Kalman
> Assignee: Flavia Rainone
> Priority: Blocker
>
> But I have another problem, when I removed trace logs from config I'm not able to run benchmark on localhost to the end, because clients starts getting :
> AMQ214013: Failed to decode packet
> java.lang.IndexOutOfBoundsException: readerIndex: 2130706436 (expected: 0 <= readerIndex <= writerIndex(2943))
> at io.netty.buffer.AbstractByteBuf.readerIndex(AbstractByteBuf.java:73)
> at io.netty.buffer.WrappedByteBuf.readerIndex(WrappedByteBuf.java:99)
> at org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readerIndex(ChannelBufferWrapper.java:405)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.decode(MessageImpl.java:1052)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.decodeFromBuffer(MessageImpl.java:459)
> at org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionReceiveMessage.decode(SessionReceiveMessage.java:94)
> at org.apache.activemq.artemis.core.protocol.ClientPacketDecoder.decode(ClientPacketDecoder.java:42)
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:371)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1374)
> at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteCha
> nnel.java:131)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at java.lang.Thread.run(Thread.java:745)
> and
> Thread-9 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
> java.lang.NumberFormatException: null
> at java.lang.Integer.parseInt(Integer.java:542)
> at java.lang.Integer.valueOf(Integer.java:766)
> at org.apache.activemq.artemis.utils.TypedProperties.getIntProperty(TypedProperties.java:280)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.getIntProperty(MessageImpl.java:811)
> at org.apache.activemq.artemis.jms.client.ActiveMQMessage.getIntProperty(ActiveMQMessage.java:578)
> at org.spec.jms.agents.SPECWorkerThread.receivedMessage(SPECWorkerThread.java:849)
> at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:820)
> at org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Thread-3 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
> javax.jms.IllegalStateException: AMQ119027: Could not find reference on consumer ID=0, messageId = 104,833 queue = 127\.0\.0\.1_VM1_SPAgent7_0.SP_CallForOffersEH_7_EHID_1PF0
> at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:410)
> at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendACK(ActiveMQSessionContext.java:461)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.acknowledge(ClientSessionImpl.java:765)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.doAck(ClientConsumerImpl.java:1212)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.flushAcks(ClientConsumerImpl.java:830)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.flushAcks(ClientSessionImpl.java:1852)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:501)
> at org.apache.activemq.artemis.core.client.impl.DelegatingSession.commit(DelegatingSession.java:159)
> at org.apache.activemq.artemis.jms.client.ActiveMQSession.commit(ActiveMQSession.java:218)
> at org.spec.jms.eventhandler.sp.SP_CallForOffersEH.handleMessage(SP_CallForOffersEH.java:306)
> at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:821)
> at org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecut
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 3 months
[JBoss JIRA] (WFLY-5480) SPEC JMS 2007 benchmark fails with AMQ214013: Failed to decode packet
by George Turner (JIRA)
[ https://issues.jboss.org/browse/WFLY-5480?page=com.atlassian.jira.plugin.... ]
George Turner commented on WFLY-5480:
-------------------------------------
I really don't have a simple way to replicate the issue. This is a large program and I cannot "share" the code. None of it is classified, but it is treated as "For OfficialUseOnly".It took us quite a while after moving to WF10 to figure out that two processes on the same topic created the exception. We have a "DataLogger" process that attaches to every topic and logs every message. If we disable that process, we have ZERO errors. When we were debugging, we saw a spot in the code (netty we think) that looked at the size of the message, and if it was too big, it was broken into smaller chunks, and the multiple consumers code broke there. The other people who have commented on this bug also claimed it as a race condition when more than one client was consuming the same message. I read the flow control documentation, and I am going to try some changes to see if I can get different results anywhere.
> SPEC JMS 2007 benchmark fails with AMQ214013: Failed to decode packet
> ---------------------------------------------------------------------
>
> Key: WFLY-5480
> URL: https://issues.jboss.org/browse/WFLY-5480
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Reporter: Ondřej Kalman
> Assignee: Flavia Rainone
> Priority: Blocker
>
> But I have another problem, when I removed trace logs from config I'm not able to run benchmark on localhost to the end, because clients starts getting :
> AMQ214013: Failed to decode packet
> java.lang.IndexOutOfBoundsException: readerIndex: 2130706436 (expected: 0 <= readerIndex <= writerIndex(2943))
> at io.netty.buffer.AbstractByteBuf.readerIndex(AbstractByteBuf.java:73)
> at io.netty.buffer.WrappedByteBuf.readerIndex(WrappedByteBuf.java:99)
> at org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readerIndex(ChannelBufferWrapper.java:405)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.decode(MessageImpl.java:1052)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.decodeFromBuffer(MessageImpl.java:459)
> at org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionReceiveMessage.decode(SessionReceiveMessage.java:94)
> at org.apache.activemq.artemis.core.protocol.ClientPacketDecoder.decode(ClientPacketDecoder.java:42)
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:371)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1374)
> at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteCha
> nnel.java:131)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at java.lang.Thread.run(Thread.java:745)
> and
> Thread-9 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
> java.lang.NumberFormatException: null
> at java.lang.Integer.parseInt(Integer.java:542)
> at java.lang.Integer.valueOf(Integer.java:766)
> at org.apache.activemq.artemis.utils.TypedProperties.getIntProperty(TypedProperties.java:280)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.getIntProperty(MessageImpl.java:811)
> at org.apache.activemq.artemis.jms.client.ActiveMQMessage.getIntProperty(ActiveMQMessage.java:578)
> at org.spec.jms.agents.SPECWorkerThread.receivedMessage(SPECWorkerThread.java:849)
> at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:820)
> at org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Thread-3 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
> javax.jms.IllegalStateException: AMQ119027: Could not find reference on consumer ID=0, messageId = 104,833 queue = 127\.0\.0\.1_VM1_SPAgent7_0.SP_CallForOffersEH_7_EHID_1PF0
> at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:410)
> at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendACK(ActiveMQSessionContext.java:461)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.acknowledge(ClientSessionImpl.java:765)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.doAck(ClientConsumerImpl.java:1212)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.flushAcks(ClientConsumerImpl.java:830)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.flushAcks(ClientSessionImpl.java:1852)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:501)
> at org.apache.activemq.artemis.core.client.impl.DelegatingSession.commit(DelegatingSession.java:159)
> at org.apache.activemq.artemis.jms.client.ActiveMQSession.commit(ActiveMQSession.java:218)
> at org.spec.jms.eventhandler.sp.SP_CallForOffersEH.handleMessage(SP_CallForOffersEH.java:306)
> at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:821)
> at org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecut
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 3 months
[JBoss JIRA] (WFLY-5480) SPEC JMS 2007 benchmark fails with AMQ214013: Failed to decode packet
by Clebert Suconic (JIRA)
[ https://issues.jboss.org/browse/WFLY-5480?page=com.atlassian.jira.plugin.... ]
Clebert Suconic commented on WFLY-5480:
---------------------------------------
you could build artemis/master and replace the jars within wildfly as a test.
Or if you have access to building wildfly, you could download wildfly, and replace the version of artemis by 1.2.0 and give it a try.
The best way though would be if you could help me replicate this issue (if you still see an issue).
> SPEC JMS 2007 benchmark fails with AMQ214013: Failed to decode packet
> ---------------------------------------------------------------------
>
> Key: WFLY-5480
> URL: https://issues.jboss.org/browse/WFLY-5480
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Reporter: Ondřej Kalman
> Assignee: Flavia Rainone
> Priority: Blocker
>
> But I have another problem, when I removed trace logs from config I'm not able to run benchmark on localhost to the end, because clients starts getting :
> AMQ214013: Failed to decode packet
> java.lang.IndexOutOfBoundsException: readerIndex: 2130706436 (expected: 0 <= readerIndex <= writerIndex(2943))
> at io.netty.buffer.AbstractByteBuf.readerIndex(AbstractByteBuf.java:73)
> at io.netty.buffer.WrappedByteBuf.readerIndex(WrappedByteBuf.java:99)
> at org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readerIndex(ChannelBufferWrapper.java:405)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.decode(MessageImpl.java:1052)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.decodeFromBuffer(MessageImpl.java:459)
> at org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionReceiveMessage.decode(SessionReceiveMessage.java:94)
> at org.apache.activemq.artemis.core.protocol.ClientPacketDecoder.decode(ClientPacketDecoder.java:42)
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:371)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1374)
> at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteCha
> nnel.java:131)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at java.lang.Thread.run(Thread.java:745)
> and
> Thread-9 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
> java.lang.NumberFormatException: null
> at java.lang.Integer.parseInt(Integer.java:542)
> at java.lang.Integer.valueOf(Integer.java:766)
> at org.apache.activemq.artemis.utils.TypedProperties.getIntProperty(TypedProperties.java:280)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.getIntProperty(MessageImpl.java:811)
> at org.apache.activemq.artemis.jms.client.ActiveMQMessage.getIntProperty(ActiveMQMessage.java:578)
> at org.spec.jms.agents.SPECWorkerThread.receivedMessage(SPECWorkerThread.java:849)
> at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:820)
> at org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Thread-3 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
> javax.jms.IllegalStateException: AMQ119027: Could not find reference on consumer ID=0, messageId = 104,833 queue = 127\.0\.0\.1_VM1_SPAgent7_0.SP_CallForOffersEH_7_EHID_1PF0
> at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:410)
> at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendACK(ActiveMQSessionContext.java:461)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.acknowledge(ClientSessionImpl.java:765)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.doAck(ClientConsumerImpl.java:1212)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.flushAcks(ClientConsumerImpl.java:830)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.flushAcks(ClientSessionImpl.java:1852)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:501)
> at org.apache.activemq.artemis.core.client.impl.DelegatingSession.commit(DelegatingSession.java:159)
> at org.apache.activemq.artemis.jms.client.ActiveMQSession.commit(ActiveMQSession.java:218)
> at org.spec.jms.eventhandler.sp.SP_CallForOffersEH.handleMessage(SP_CallForOffersEH.java:306)
> at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:821)
> at org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecut
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 3 months
[JBoss JIRA] (WFLY-5480) SPEC JMS 2007 benchmark fails with AMQ214013: Failed to decode packet
by George Turner (JIRA)
[ https://issues.jboss.org/browse/WFLY-5480?page=com.atlassian.jira.plugin.... ]
George Turner commented on WFLY-5480:
-------------------------------------
Here is the messaging section from the standalone-full.xml. I added [consumer-window-size="10485760" producer-window-size="10485760"] to the RemoteConnectionFactory and it did not seem to help.
<subsystem xmlns="urn:jboss:domain:messaging-activemq:1.0"> <server name="default" persistence-enabled="false"> <security enabled="false"/> <security-setting name="#"> <role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/> </security-setting> <address-setting name="#" dead-letter-address="jms.queue.DLQ" expiry-address="jms.queue.ExpiryQueue" max-size-bytes="104857600" page-size-bytes="2097152" message-counter-history-day-limit="10" address-full-policy="BLOCK"/> <http-connector name="http-connector" socket-binding="http" endpoint="http-acceptor"/> <http-connector name="http-connector-throughput" socket-binding="http" endpoint="http-acceptor-throughput"> <param name="batch-delay" value="50"/> </http-connector> <in-vm-connector name="in-vm" server-id="0"/> <http-acceptor name="http-acceptor" http-listener="default"/> <http-acceptor name="http-acceptor-throughput" http-listener="default"> <param name="batch-delay" value="50"/> <param name="direct-deliver" value="false"/> </http-acceptor> <in-vm-acceptor name="in-vm" server-id="0"/> <jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/> <jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/> <jms-topic name="AtmosphericParameters" entries="java:jboss/exported/topic/AtmosphericParameters"/> <jms-topic name="C2SimInput" entries="java:jboss/exported/topic/C2SimInput"/> <jms-topic name="C2SimNotify" entries="java:jboss/exported/topic/C2SimNotify"/> <jms-topic name="C2SimCatalogUpdate" entries="java:jboss/exported/topic/C2SimCatalogUpdate"/> <jms-topic name="CalibrationThresholds" entries="java:jboss/exported/topic/CalibrationThresholds"/><jms-topic name="TracksIn" entries="java:jboss/exported/topic/TracksIn"/><jms-topic name="UncalibratedCuedSearchRequestMsg" entries="java:jboss/exported/topic/UncalibratedCuedSearchRequestMsg"/><connection-factory name="InVmConnectionFactory" connectors="in-vm" entries="java:/ConnectionFactory"/><connection-factory name="RemoteConnectionFactory" connectors="http-connector" entries="java:jboss/exported/jms/RemoteConnectionFactory" consumer-window-size="10485760" producer-window-size="10485760"/><pooled-connection-factory name="activemq-ra" transaction="xa" connectors="in-vm" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory"/>
> SPEC JMS 2007 benchmark fails with AMQ214013: Failed to decode packet
> ---------------------------------------------------------------------
>
> Key: WFLY-5480
> URL: https://issues.jboss.org/browse/WFLY-5480
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Reporter: Ondřej Kalman
> Assignee: Flavia Rainone
> Priority: Blocker
>
> But I have another problem, when I removed trace logs from config I'm not able to run benchmark on localhost to the end, because clients starts getting :
> AMQ214013: Failed to decode packet
> java.lang.IndexOutOfBoundsException: readerIndex: 2130706436 (expected: 0 <= readerIndex <= writerIndex(2943))
> at io.netty.buffer.AbstractByteBuf.readerIndex(AbstractByteBuf.java:73)
> at io.netty.buffer.WrappedByteBuf.readerIndex(WrappedByteBuf.java:99)
> at org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readerIndex(ChannelBufferWrapper.java:405)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.decode(MessageImpl.java:1052)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.decodeFromBuffer(MessageImpl.java:459)
> at org.apache.activemq.artemis.core.protocol.core.impl.wireformat.SessionReceiveMessage.decode(SessionReceiveMessage.java:94)
> at org.apache.activemq.artemis.core.protocol.ClientPacketDecoder.decode(ClientPacketDecoder.java:42)
> at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:371)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1374)
> at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
> at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
> at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteCha
> nnel.java:131)
> at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at java.lang.Thread.run(Thread.java:745)
> and
> Thread-9 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
> java.lang.NumberFormatException: null
> at java.lang.Integer.parseInt(Integer.java:542)
> at java.lang.Integer.valueOf(Integer.java:766)
> at org.apache.activemq.artemis.utils.TypedProperties.getIntProperty(TypedProperties.java:280)
> at org.apache.activemq.artemis.core.message.impl.MessageImpl.getIntProperty(MessageImpl.java:811)
> at org.apache.activemq.artemis.jms.client.ActiveMQMessage.getIntProperty(ActiveMQMessage.java:578)
> at org.spec.jms.agents.SPECWorkerThread.receivedMessage(SPECWorkerThread.java:849)
> at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:820)
> at org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Thread-3 (ActiveMQ-client-global-threads-1266561810): Uncaught exception.
> javax.jms.IllegalStateException: AMQ119027: Could not find reference on consumer ID=0, messageId = 104,833 queue = 127\.0\.0\.1_VM1_SPAgent7_0.SP_CallForOffersEH_7_EHID_1PF0
> at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:410)
> at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendACK(ActiveMQSessionContext.java:461)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.acknowledge(ClientSessionImpl.java:765)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.doAck(ClientConsumerImpl.java:1212)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.flushAcks(ClientConsumerImpl.java:830)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.flushAcks(ClientSessionImpl.java:1852)
> at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:501)
> at org.apache.activemq.artemis.core.client.impl.DelegatingSession.commit(DelegatingSession.java:159)
> at org.apache.activemq.artemis.jms.client.ActiveMQSession.commit(ActiveMQSession.java:218)
> at org.spec.jms.eventhandler.sp.SP_CallForOffersEH.handleMessage(SP_CallForOffersEH.java:306)
> at org.spec.jms.agents.SPECWorkerThread.onMessage(SPECWorkerThread.java:821)
> at org.apache.activemq.artemis.jms.client.JMSMessageListenerWrapper.onMessage(JMSMessageListenerWrapper.java:100)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:1089)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:47)
> at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1224)
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecut
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 3 months
[JBoss JIRA] (WFLY-459) javassist should also be exported to Hibernate (native) applications
by Scott Marlow (JIRA)
[ https://issues.jboss.org/browse/WFLY-459?page=com.atlassian.jira.plugin.s... ]
Scott Marlow commented on WFLY-459:
-----------------------------------
Some example bytecode output from JavassistProxyFactory [https://gist.github.com/scottmarlow/dc7ebfea654984f84e2e] that references the Javassist classes.
> javassist should also be exported to Hibernate (native) applications
> --------------------------------------------------------------------
>
> Key: WFLY-459
> URL: https://issues.jboss.org/browse/WFLY-459
> Project: WildFly
> Issue Type: Task
> Components: JPA / Hibernate
> Reporter: Scott Marlow
> Assignee: Scott Marlow
> Fix For: 10.0.0.Final
>
>
> Originally this jira was to stop exporting the javassist module to JPA deployments but it turns out that is fine to do. In addition, native Hibernate applications that depend on Hibernate, should also get the javassist dependency.
> # native Hibernate applications will get the javassist dependency via the application dependency on org.hibernate.
> # container managed JPA applications will get the javassist dependency via org.jboss.as.jpa.processor.JPADependencyProcessor (JPA deployer).
> # container managed JPA applications that embed their own copy of Hibernate jars will also get the javassist dependency via org.jboss.as.jpa.processor.JPADependencyProcessor (JPA deployer).
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 3 months
