I have undertow embedded into my application and for the most part its working great, but unfortunately I've found a memory leak. The application its self creates an Undertow server using SSL and then adds servlets and filters. There are no custom handlers.
Throughout the logs I get the following NPEs:
2016-01-14 09:50:04,681][XNIO-2 task-28] ERROR request - UT005071: Undertow request failed HttpServerExchange{ GET /wp-content/themes/Nexus/style.css request {Accept=[text/css, */*], Connection=[Keep-Alive], Accept-Language=[en-US,en;q=0.5], Accept-Encoding=[gzip, deflate], Cookie=[nowwwsession=eyJpdiI6Ik5hV0xCWWR5VExzUEEzRTBVNUFKUkFcdTAwM2RcdTAwM2QiLCJlbmNyeXB0ZWRSZXF1ZXN0IjoiS2t0TENaVGZFb1NWNjlyVXB4aUxuQ2RHRmlTZzVDWm0veTFXLytXWlBQMEhpMDZKK3FaQWpQYnBGY0RnMW15RCJ9; nvers-prod-session=eyJpdiI6Ikd2Wk5oTDVGbjdSTElGejlQSzFsalFcdTAwM2RcdTAwM2QiLCJlbmNyeXB0ZWRSZXF1ZXN0IjoiN2VldzBiT1lIWkZNaVgwTzV5b2tUSVNnaWwzTC9tRHpmeEdDOFZiQzh2a1RZN29ScElqbHFyZys3dmZMWUNhRSJ9], Referer=[
https://www.nvers.org/the-team/], User-Agent=[Mozilla/5.0 (Windows NT 10.0; Trident/7.0; MALC; rv:11.0) like Gecko], Host=[
www.nvers.org]} response {Connection=[close], Last-Modified=[Thu, 26 Mar 2015 18:45:16 GMT], Server=[Apache/2.2.15 (CentOS)], Content-Type=[text/css], Content-Language=[en-US], Accept-Ranges=[bytes], Date=[Thu, 14 Jan 2016 14:37:10 GMT]}}
java.lang.NullPointerException
at io.undertow.server.protocol.http.HttpResponseConduit.flush(HttpResponseConduit.java:713)
at io.undertow.conduits.FinishableStreamSinkConduit.flush(FinishableStreamSinkConduit.java:83)
at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162)
at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:119)
at org.xnio.channels.Channels.flushBlocking(Channels.java:63)
at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:609)
at io.undertow.servlet.spec.HttpServletResponseImpl.closeStreamAndWriter(HttpServletResponseImpl.java:476)
at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:560)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:331)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:263)
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:174)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Also, I see the following in System.out:
Exception in thread "XNIO-2 task-18" java.lang.NullPointerException
at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:605)
at io.undertow.conduits.ChunkedStreamSinkConduit.flush(ChunkedStreamSinkConduit.java:267)
at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162)
at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:119)
at io.undertow.server.HttpServerExchange.closeAndFlushResponse(HttpServerExchange.java:1652)
at io.undertow.server.HttpServerExchange.endExchange(HttpServerExchange.java:1630)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:226)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
After taking a heap dump and analyzing, I'm finding that a single thread is holding all the objects:
The thread org.xnio.nio.WorkerThread @ 0xc48cac70 XNIO-2 I/O-4 keeps local variables with total size 578,543,888 (95.77%) bytes.
The memory is accumulated in one instance of "java.lang.Object[]" loaded by "<system class loader>".
XNIO-2 I/O-4
at sun.security.ssl.SSLEngineImpl.unwrap(Ljava/nio/ByteBuffer;[Ljava/nio/ByteBuffer;II)Ljavax/net/ssl/SSLEngineResult; (SSLEngineImpl.java:781)
at io.undertow.protocols.ssl.SslConduit.doUnwrap([Ljava/nio/ByteBuffer;II)J (SslConduit.java:682)
at io.undertow.protocols.ssl.SslConduit.read(Ljava/nio/ByteBuffer;)I (SslConduit.java:525)
at org.xnio.conduits.ConduitStreamSourceChannel.read(Ljava/nio/ByteBuffer;)I (ConduitStreamSourceChannel.java:127)
at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(Lorg/xnio/conduits/ConduitStreamSourceChannel;)V (HttpReadListener.java:149)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(Lorg/xnio/conduits/ConduitStreamSourceChannel;)V (HttpReadListener.java:127)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(Ljava/nio/channels/Channel;)V (HttpReadListener.java:56)
at org.xnio.ChannelListeners.invokeChannelListener(Ljava/nio/channels/Channel;Lorg/xnio/ChannelListener;)Z (ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady()V (ReadReadyHandler.java:66)
at io.undertow.protocols.ssl.SslConduit$SslReadReadyHandler.readReady()V (SslConduit.java:1054)
at io.undertow.protocols.ssl.SslConduit$1.run()V (SslConduit.java:225)
at org.xnio.nio.WorkerThread.safeRun(Ljava/lang/Runnable;)V (WorkerThread.java:580)
at org.xnio.nio.WorkerThread.run()V (WorkerThread.java:464)
Any help would be appreciated. We didn't have any memory leak issues with Jetty which is what we used prior so I don't think its an issue with our internal code triggering this.
Thanks
Marc Boorshtein
CTO Tremolo Security