[jboss-jira] [JBoss JIRA] (WFLY-4790) Getting DirectBuffer OOM when sending fragmented binary message to websocket endpoint

Getting DirectBuffer OOM when sending fragmented binary message to websocket endpoint ------------------------------------------------------------------------------------- Key: WFLY-4790 URL: https://issues.jboss.org/browse/WFLY-4790 Project: WildFly Issue Type: Bug Components: Web (Undertow) Affects Versions: 10.0.0.Alpha3 Reporter: Radim Hatlapatka Assignee: Stuart Douglas Priority: Critical Fix For: 10.0.0.Alpha5 When sending fragmented binary message (message with message payload of length 4 * 2**20 (4M). Sent out in fragments of 64). The server throws {{java.lang.OutOfMemoryError: Direct buffer memory}} [1] The memory for direct buffer by default depends on the size set by -Xmx, which is in EAP 7.0.0.DR4 by default set to -Xmx512m. Increasing it just increases the time before the limit is hit (it is enough to send those messages multiple times to hit the limit again). I believe the issue is similar to the one for EAP 6.4: [https://bugzilla.redhat.com/show_bug.cgi?id=1223708] [1] {noformat} 15:10:55,463 ERROR [org.xnio.listener] (default I/O-1) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:658) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) at org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:57) at org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:55) at org.xnio.ByteBufferSlicePool.allocate(ByteBufferSlicePool.java:143) at io.undertow.websockets.core.BufferedBinaryMessage$1.handleEvent(BufferedBinaryMessage.java:106) at io.undertow.websockets.core.BufferedBinaryMessage$1.handleEvent(BufferedBinaryMessage.java:97) at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel$1.run(AbstractFramedStreamSourceChannel.java:264) at org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:560) at org.xnio.nio.WorkerThread.run(WorkerThread.java:462) {noformat}