Use MessageToByteEncoder to avoid ByteBuf leak w/ exceptions ------------------------------------------------------------ Key: ISPN-5576 URL: https://issues.jboss.org/browse/ISPN-5576 Project: Infinispan Issue Type: Bug Components: Remote Protocols Affects Versions: 7.2.3.Final Reporter: Galder Zamarreño Assignee: Galder Zamarreño Fix For: 8.0.0.Beta1, 7.2.4.Final, 8.0.0.Final Encoder implementation allocates buffers but does not release them in case of exception before the encoded buffer is added to the response list. As a result, if encoding faces exceptions, it could leak byte buffers and show messages like this: {code} [io.netty.util.ResourceLeakDetector] LEAK: ByteBuf.release() was not called before it's garbage-collected. Enable advanced leak reporting to find out where the leak occurred. To enable advanced leak reporting, specify the JVM option '-Dio.netty.leakDetectionLevel=advanced' or call ResourceLeakDetector.setLevel() LEAK: ByteBuf.release() was not called before it's garbage-collected. Recent access records: 0 Created at: io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:259) io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155) io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:141) io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:75) org.infinispan.server.hotrod.HotRodEncoder.encode(HotRodEncoder.scala:29) io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:89) io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:633) io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:691) io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:626) io.netty.handler.timeout.IdleStateHandler.write(IdleStateHandler.java:266) io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:633) io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:691) io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:681) io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:716) io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:954) io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:243) org.infinispan.server.core.AbstractProtocolDecoder.writeResponse(AbstractProtocolDecoder.scala:220) org.infinispan.server.hotrod.HotRodDecoder.customDecodeHeader(HotRodDecoder.scala:153) org.infinispan.server.core.AbstractProtocolDecoder.org$infinispan$server$core$AbstractProtocolDecoder$$decodeHeader(AbstractProtocolDecoder.scala:137) org.infinispan.server.core.AbstractProtocolDecoder$$anon$2.run(AbstractProtocolDecoder.scala:98) org.infinispan.server.core.AbstractProtocolDecoder$$anon$2.run(AbstractProtocolDecoder.scala:95) org.infinispan.security.Security.doAs(Security.java:143) org.infinispan.server.core.AbstractProtocolDecoder.secureDecodeDispatch(AbstractProtocolDecoder.scala:95) org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:59) io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:370) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:168) org.infinispan.server.core.AbstractProtocolDecoder.channelRead(AbstractProtocolDecoder.scala:459) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) java.lang.Thread.run(Thread.java:745) {code} Also, encoder implementation does not log exceptions reported at encoding time, so exceptions like this can only be noticed via instrumentation.