[
https://jira.jboss.org/jira/browse/ISPN-391?page=com.atlassian.jira.plugi...
]
Galder Zamarreno commented on ISPN-391:
---------------------------------------
I'm attaching a test that tries to replicate this issue in a unit test and I suspect
that Mircea's suspicions are correct cos I'm getting some very funny NPEs such
as:
2010-04-06 16:33:47,426 ERROR [AbstractProtocolDecoder$] (New I/O server worker #1-3)
Exception reported
java.lang.NullPointerException
at
org.infinispan.server.core.transport.netty.ChannelBufferAdapter.readableBytes(ChannelBufferAdapter.scala:30)
at
org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:31)
at
org.infinispan.server.core.transport.netty.DecoderAdapter.decode(DecoderAdapter.scala:19)
at
org.infinispan.server.core.transport.netty.DecoderAdapter.decode(DecoderAdapter.scala:13)
at
org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:461)
at
org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:434)
It would appear that under a multi-threaded environment, the buffer that's passed to
the decode() implementer can, under multi-threaded situations, be null.
Hot Rod/Memcached decoders fail to decode properly with multiple
worker threads
-------------------------------------------------------------------------------
Key: ISPN-391
URL:
https://jira.jboss.org/jira/browse/ISPN-391
Project: Infinispan
Issue Type: Bug
Components: Cache Server
Affects Versions: 4.1.0.ALPHA1, 4.1.0.ALPHA2
Reporter: Galder Zamarreno
Assignee: Galder Zamarreno
Fix For: 4.1.0.BETA1
There is an frequent(almost on every run) intermittent failure on one of my integration
tests[1].
For the current release I found a workaround for it, by fixing the workers thread pool in
Netty transport to 1. Just rollback my commit on NettyTransport (and re- mvn install
server/core, I always use to forget that ) From my investigation, I think the problem is
caused by the following scenario:
- AbstractProtocolDecoder.decode is being called upon a request from client, the code is
being processed by worker thread ONE
- ONE reads the header, but no more information is available and it blocks on some read
(TCP splits the request from server into multiple packets and NIO delivers them in
multiple calls)
- AbstractProtocolDecoder.decode is being called again, with the rest of bytes from the
same request, but within thread TWO
- thread TWO tries to read magic (skips bytes) and thread ONE hangs, waiting for more
information.
If this is the problem , a solution would be to confine each channel to a single
processing thread, as netty might use multiple processing threads for bytes originating
from the same channel.
[1] [trunk/client/hotrod-client ]$ mvn test -Dtest=HotRodIntegrationTest
p.s. This might affect memcached as well, double check.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira