[jboss-jira] [JBoss JIRA] Commented: (JBVFS-159) Native memory leak due to ZipEntryInputStream

[Marko Strukelj (JIRA)]

    [ https://jira.jboss.org/browse/JBVFS-159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12536436#action_12536436 ] 

Marko Strukelj commented on JBVFS-159:
--------------------------------------

To clarify, my proposal regarding finalize() method is not really about any memory leak, but about garbage collection dynamics which is affected by finalizers, and causes such objects to take more gc cycles to remove.

As far as memory leak is concerned ... 

I tried some tests with, and without the proposed removal of ZipEntryInputStream, using vfs 2.1.2.GA. I am unable to see any noticable difference in memory usage between the two - which means the difference is less than the variance. My test works in isolation from jboss-as. It uses VFS directly to scan and fully read the content of all the archives in jbossas deploy directory, where I have some extra ears, some 50Mb big. In this scenario the memory goes up to about 150M in both cases.

If all the code that uses resource/class streams properly closes streams after reading or simply just fully reads the streams, then there should be no memory leaking here - and I can't find one. If there are outstanding open streams on the zip file, then of course it would be wrong to forcefully close it (equivallent of pulling the rug from under the open stream - causing a potential IOException). We shouldn't blame VFS for such leaks. Again, the function of ZipEntryInputStream is precisely to support timely, and correct release of ZipFile objects - therefore to release memory, and file locks as soon as possible.

I'm attaching the test. Place it in jboss_vfs/src/test/java/org/jboss/test/virtual/test.
Adjust the value of DIR variable to point to your deploy dir or wherever you have many java archive files.

Run the test with:

mvn test -Dtest=VFSZipMemoryTestCase

You can tweak memory options using:

mvn test -Dtest=VFSZipMemoryTestCase "-DargLine=-server -Xss128k -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC"


Use jvisualvm in parallel, connect to SurefireBooter process when it appears in the list of processes, observe memory usage, do some memory profiling, some snapshots ... During the run you can see instances of ZipEntryInputStream being created. But at the end of the run their number is zero - meaning they were all released and GC-ed. Ergo - with proper usage there is no memory leak.

We need to differentiate between a memory leak, and a runtime memory footprint.

There doesn't seem to be any memory leak, and runtime footprint for the isolated test is indistinguishable between current code and your proposed fix.


My hunch is that in your case you have some code somewhere - maybe some third party library, that holds on to the ZipEntryInputStream preventing the release of underlying ZipFileWrapper, and transitively the ZipFile - the one that actually consumes memory, as everything else are just lightweight wrappers around it.


> Native memory leak due to ZipEntryInputStream
> ---------------------------------------------
>
>                 Key: JBVFS-159
>                 URL: https://jira.jboss.org/browse/JBVFS-159
>             Project: JBoss VFS
>          Issue Type: Bug
>      Security Level: Public(Everyone can see) 
>    Affects Versions: 2.1.2.GA
>         Environment: Redhat (not sure what version) 2.6.9-78.ELsmp
> JBoss 5.1.0.GA
> JDK 1.6.0_20
> JVM parameter:
> -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -verbose:gc -Dfile.encoding=iso-8859-1 -server -Djava.net.preferIPv4Stack=true -Doracle.jdbc.V8Compatible=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dnetworkaddress.cache.ttl=300 -Xss128k -Xmn500m -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -XX:+PrintGCDetails -XX:PermSize=256m -XX:MaxPermSize=256m -Xms1500m -Xmx1500m -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC 
>            Reporter: Samuel Cai
>            Assignee: Marko Strukelj
>
> We used to use JBoss 4.2.1.GA and JDK 1.6.0_11, and trying JBoss 5.1.0.GA and JDK 1.6.0_20 these days.
> I found the process size is more larger than before, 2.5G~2.9G compared to 1.9G.
> I was thinking this was a bug of JBoss AS, then filed https://jira.jboss.org/browse/JBAS-8066
> After these days investigation, I think this is a memory leak in VFS, maybe only happen on our specific environment.
> I tried a change on class org.jboss.virtual.plugins.context.zip.ZipFileWrapper, method openStream:
> From:
> ZipEntryInputStream zis = new ZipEntryInputStream(this, is);
> return zis;
> To:
> //ZipEntryInputStream zis = new ZipEntryInputStream(this, is);
> //return zis;
> return is;
> That is, don't use ZipEntryInputStream, let any class/method invoking openStream to close zipFile's inputStream immediatelly.
> ZipFile will be in open status, but all steams will be closed well.
> This makes the process size down to same as JBoss 4's. I tried going through first 3 pages of site, no problems. May need QA team to test more.
> Btw, I tried updating VFS to 2.1.3.SP1/2.2.0.M4/3.0.0.CR5, first two have same size issue, third one couldn't start.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira