[jboss-jira] [JBoss JIRA] Created: (JBPORTAL-2466) Native heap memory leak when rendering Portlet URLs causes JVM crash

[Patrik Gottfridsson (JIRA)]

Native heap memory leak when rendering Portlet URLs causes JVM crash
--------------------------------------------------------------------

                 Key: JBPORTAL-2466
                 URL: https://jira.jboss.org/jira/browse/JBPORTAL-2466
             Project: JBoss Portal
          Issue Type: Bug
      Security Level: Public (Everyone can see)
          Components: Portal Core
    Affects Versions: 2.7.2 Final
         Environment: Operating system: Linux 2.6.18-164.el5PAE #1 SMP Thu Sep 3 04:10:44 EDT 2009 i686 i686 i386 GNU/Linux
JVM: Sun JRE 6.0_16-b01
Java heap settings: -Xms1024m -Xmx1024m -XX:+UseParallelGC -XX:MaxPermSize=256m
Running JBoss Portal using "Simple Portal" taglibs in Tomcat 6.0.20

            Reporter: Patrik Gottfridsson


Under moderate load testing of a page containing thirtysome Portlet URLs,  the JVM crashes quickly with various variants of the following stderr message:

# A fatal error has been detected by the Java Runtime Environment:
#
# java.lang.OutOfMemoryError: requested 70968 bytes for Chunk::new. Out of swap space?
#
#  Internal Error (allocation.cpp:215), pid=24331, tid=1870162832
#  Error: Chunk::new
#
# JRE version: 6.0_16-b01
# Java VM: Java HotSpot(TM) Server VM (14.2-b01 mixed mode linux-x86 )
# An error report file with more information is saved as:
..

Lowering the Java heap size to 1/4 of the original size avoided the problem (but was for other reasons not acceptable). The problem was thus avoided when garbage collection was more frequent. This behaviour, in combination with the error message above, indicated that this was in fact a native heap leak and that this native memory was being freed when the Java garbage collector cleaned up Java objects and triggered their finalizers.

One common case of native heap leak problems is using the GZipOutputStream for compression. The GZipOutputStream is a thin wrapper for native ZLIB resources which consume a lot of native memory. Creating a lot of GZipOuputStream objects without closing them can thus crash the JVM since ZLIB fills upp the native heap before the wrapping GZipOutputStream objects fill upp the Java heap and trigger a garbage collection.

Looking at the JBoss Portal sources, GzipOutputStream is used to compress parameters in Portlet URLs. After compression, GZipOutstream.finish() is invoked but never GZipOutstream.close(). This leaves the ZLIB native resources hanging and causes a native heap leak.

I manually patched 

org.jboss.portal.common.io.SerializationFilter 

to force a GZipOutstream.close(). This solved the problem and the application now behaves nicely under load also with large heap settings. It would be great to have this included in an official patch by changing

public <T> void serialize(Serialization<T> serialization, T t, OutputStream out) throws IllegalArgumentException, IOException
      {
         GZIPOutputStream zos = new GZIPOutputStream(out);
         serialization.serialize(t, zos);
         zos.finish();
      }


to

public <T> void serialize(Serialization<T> serialization, T t, OutputStream out) throws IllegalArgumentException, IOException
      {
         GZIPOutputStream zos = null;
	   try {
	       zos = new GZIPOutputStream(out);
             serialization.serialize(t, zos);
             zos.finish();
         } finally {
             if (zos != null) {
                 zos.close();
             }
         }
      }

or similar.


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira