4:59 a.m.
[
https://jira.jboss.org/jira/browse/JBAS-7468?page=com.atlassian.jira.plug...
]
Michal Borowiecki commented on JBAS-7468:
-----------------------------------------
Thanks for the workaround.
This issue can hit you quite suddenly. Our monitoring kept showing 100MB of free heap
space and then OutOfMemoryErrors started appearing, monitoring still showing 100MB of free
heap.
"controlFlags" must have grown to over 100MB by that time, and on .add() a new
enlarged copy of the underlying array was being allocated resulting in OutOfMemoryError
while heap usage remained at 100MB below max.
Here is the stack trace for reference, so that someone can google it when it hits them ;)
2010-03-31 20:13:06,791 ERROR [org.apache.catalina.connector.CoyoteAdapter] An exception
or error occurred in the container during the request processing
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2734)
at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
at java.util.ArrayList.add(ArrayList.java:351)
at
org.jboss.security.plugins.authorization.JBossAuthorizationContext.initializeModules(JBossAuthorizationContext.java:198)
at
org.jboss.security.plugins.authorization.JBossAuthorizationContext.authorize(JBossAuthorizationContext.java:139)
at
org.jboss.security.plugins.JBossAuthorizationManager.internalAuthorization(JBossAuthorizationManager.java:474)
at
org.jboss.security.plugins.JBossAuthorizationManager.authorize(JBossAuthorizationManager.java:124)
at
org.jboss.security.plugins.javaee.WebAuthorizationHelper.hasUserDataPermission(WebAuthorizationHelper.java:201)
at
org.jboss.web.tomcat.security.JBossWebRealm.hasUserDataPermission(JBossWebRealm.java:643)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:461)
at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
at
org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
at
org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:383)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
at
org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:619)
Memory leak in
org.jboss.security.plugins.authorization.JBossAuthorizationContext
---------------------------------------------------------------------------------
Key: JBAS-7468
URL: https://jira.jboss.org/jira/browse/JBAS-7468
Project: JBoss Application Server
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Security
Affects Versions: JBossAS-5.1.0.GA
Environment: JBoss Version: jboss-5.1.0.GA, OS: Linux (2.6.18-164.el5),
Architecture: amd64 64bit, JVM: Java HotSpot(TM) 64-Bit Server VM (14.0-b16, mixed mode)
Reporter: Ganesh Ingle
Assignee: Anil Saldhana
Fix For: JBossAS-6.0.0.M1
Attachments: HeapMemoryOldGen.png, JBossAuthorizationContext.java,
JBossAuthorizationContext_MemLeak.png
Our use case (only security related portion is mentioned here):
Axis 1.4 webservice, standard J2EE declarative security through WEB-INF/web.xml, a http
client sends soap request and BASIC auth information, the JBoss server performs
authentication and authorization as per WEB-INF/web.xml configuration.
We did a performance/stability test on above web service. After 8.5 million requests the
server gone out of memory. We did heap dump analysis using VisualVM tool and found that
the class org.jboss.security.plugins.authorization.JBossAuthorizationContext is consuming
most of the memory. This class has a memer array named "controlFlags", this
array was showing 25.7 million ControlFlag entries.
When we investigated the code we found that there is one instance of
JBossAuthorizationManager per security domain and this manager has one instance of
JBossAuthorizationContext. For every authorization the JBossAuthorizationContext
initializes authorization modules and pushes their control flags (instances of class
ControlFlag) in member arrays. When the authorization is complete, a commit/abort is
invoked on all modules and finally the "modules" array is cleared. However, the
"controlFlags" array is not cleared. We checked the entire class, this array
never gets cleared.
We changed the code to clear both "modules" and "controlFlags" array
in a finally block in method JBossAuthorizationContext.authorize(final Resource resource,
final Subject subject, final RoleGroup callerRoles). We ran a 50million test after this
fix, the test was successful which proves the fix worked.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira