[jboss-jira] [JBoss JIRA] (WFLY-4696) OutOfMemory DirectByteBuffer XNIO

Mon Jul 13 13:08:05 EDT 2015

    [ https://issues.jboss.org/browse/WFLY-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088985#comment-13088985 ] 

Carlos Rodríguez Aguado edited comment on WFLY-4696 at 7/13/15 1:07 PM:
------------------------------------------------------------------------

The problem has not been solved. 

The memory leak continues. *We have a attached a video showing the problem: !wfly.mp4!*

In the video we had just started WildFly 8.2 Final. We opened also jvisualvm to monitorize the direct buffer pools, where memory is not being released. We entered to our web application (servlets, jsps, AJAX, jQuery) and began to  reproduce the problem just refreshing/reloading (F5) the browser very fast. The server responses but channel is broken before the response arrives to the client (browser). After a few seconds direct buffers pool memory had increased and after executing a GC the memory is not released in the direct buffer pool (only released in java heap).

We have been working a lot of time in this problem. We have been investigating where the memory was not released. We found that 3 direct buffer pools of class org.xnio.ssl.JsseSslConduitEngine were not released:

  /** The buffer into which incoming SSL data is written. */
  private final Pooled<ByteBuffer> receiveBuffer;

  /** The buffer from which outbound SSL data is sent. */
  private final Pooled<ByteBuffer> sendBuffer;

  /** The buffer into which inbound clear data is written. */
  private final Pooled<ByteBuffer> readBuffer;

We investigated how to release this buffers and finally we changed the code of write() methods in class io.undertow.server.protocol.http.HttpResponseConduit adding a try/catch to call to a generic method to free HttpServerExchange:

   public int write(final ByteBuffer src) throws IOException
  {
    int oldState = this.state;
    int state = oldState & MASK_STATE;
    int alreadyWritten = 0;
    int originalRemaining = -1;
    {color:red}try
    {{color}
      if (state != 0)
      {
        originalRemaining = src.remaining();
        state = processWrite(state, src, -1, -1);
        if (state != 0)
        {
          return 0;
        }
        alreadyWritten = originalRemaining - src.remaining();
        if (allAreSet(oldState, FLAG_SHUTDOWN))
        {
          next.terminateWrites();
          throw new ClosedChannelException();
        }
      }
      if (alreadyWritten != originalRemaining)
      {
        return next.write(src) + alreadyWritten;
      }
      return alreadyWritten;
	{color:red}}
    catch (IOException | RuntimeException re)
    {

      try
      {
        exchange.getRequestChannel().suspendReads();
      }
      catch (Exception e)
      {
      }
      try
      {
        exchange.getResponseChannel().shutdownWrites();
      }
      catch (Exception e)
      {
      }
      try
      {
        ((HttpServerConnection) exchange.getConnection()).getChannel().getSinkChannel().shutdownWrites();
      }
      catch (Exception e)
      {
      }
      IoUtils.safeClose(exchange.getConnection());

      throw re;
    }{color}
    finally
    {
      this.state = oldState & ~MASK_STATE | state;
    }
  }

  public long write(final ByteBuffer[] srcs) throws IOException
  {
    return write(srcs, 0, srcs.length);
  }

  public long write(final ByteBuffer[] srcs, final int offset, final int length) throws IOException
  {
    if (length == 0)
    {
      return 0L;
    }
    int oldVal = state;
    int state = oldVal & MASK_STATE;
    {color:red}try
    {{color}
      if (state != 0)
      {
        long rem = Buffers.remaining(srcs, offset, length);
        state = processWrite(state, srcs, offset, length);

        long ret = rem - Buffers.remaining(srcs, offset, length);
        if (state != 0)
        {
          return ret;
        }
        if (allAreSet(oldVal, FLAG_SHUTDOWN))
        {
          next.terminateWrites();
          throw new ClosedChannelException();
        }
        //we don't attempt to write again
        return ret;
      }
      return length == 1 ? next.write(srcs[offset]) : next.write(srcs, offset, length);
    {color:red}}
    catch (IOException | RuntimeException re)
    {

      try
      {
        exchange.getRequestChannel().suspendReads();
      }
      catch (Exception e)
      {
      }
      try
      {
        exchange.getResponseChannel().shutdownWrites();
      }
      catch (Exception e)
      {
      }
      try
      {
        ((HttpServerConnection) exchange.getConnection()).getChannel().getSinkChannel().shutdownWrites();
      }
      catch (Exception e)
      {
      }
      IoUtils.safeClose(exchange.getConnection());

      throw re;
    }{color}
    finally
    {
      this.state = oldVal & ~MASK_STATE | state;
    }
  }

We tested these changes and obtained an improvement:
* Before: OutOfMemory was reproduced in 4 or 5 days
* After: More than 2 weeks working and OutOfMemory has not been reproduced!. *But the memory is still increasing!!* Maybe exists another memory leak or the solution we applied is not the right one.

We tried also with WildFly 9 CR1 but the same problem occurs.

We don't understand how it is possible that this problem don't occur to more WildFly users.

Please, can you help us?.

was (Author: carlosra85):
The problem has not been solved. 

The memory leak continues. I have a attached a video showing the problem.

In the video we had just started WildFly 8.2 Final. We opened also jvisualvm to monitorize the direct buffer pools, where memory is not being released. We entered to our web application (servlets, jsps, AJAX, jQuery) and began to  reproduce the problem just refreshing/reloading (F5) the browser very fast. The server responses but channel is broken before the response arrives to the client (browser). After a few seconds direct buffers pool memory had increased and after executing a GC the memory is not released in the direct buffer pool (only released in java heap).