[jboss-jira] [JBoss JIRA] (WFCORE-695) OutOfMemoryErrors when adding large deployment with audit logging enabled

[Brian Stansberry (JIRA)]

     [ https://issues.jboss.org/browse/WFCORE-695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brian Stansberry reassigned WFCORE-695:
---------------------------------------

    Component/s: CLI
       Assignee: Alexey Loubyansky


[~aloubyansky] I don't think we can/should directly fix this in the audit logging layer; that's just a generic write of the contents of the mgmt op and fixing this would basically involve hard-coding a very specific deviation from the normal handling.

I think the main issue here is the CLI 'deploy' commands are shipping the deployment bytes to the server via base-64 encoded DMR (the "bytes" param to the op) instead of attaching a stream to the op.

I suspect it's done this way because attaching a stream is tricky since you need an Operation object scoped to whatever set of operations need to get run in a single request.

> OutOfMemoryErrors when adding large deployment with audit logging enabled
> -------------------------------------------------------------------------
>
>                 Key: WFCORE-695
>                 URL: https://issues.jboss.org/browse/WFCORE-695
>             Project: WildFly Core
>          Issue Type: Bug
>          Components: CLI, Domain Management
>    Affects Versions: 1.0.0.CR4
>            Reporter: James Livingston
>            Assignee: Alexey Loubyansky
>
> When performing a deployment operation, audit logging will include the deployment content in the audit record (serialised as text). Since JsonAuditLogItemFormatter.createRecordText() creates the string in memory, large deployment will lead to an OutOfMemoryError such as
> java.lang.OutOfMemoryError: Java heap space
>         at java.util.Arrays.copyOf(Arrays.java:2367) [rt.jar:1.7.0_71]
>         at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130) [rt.jar:1.7.0_71]
>         at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114) [rt.jar:1.7.0_71]
>         at java.lang.AbstractStringBuilder.appendCodePoint(AbstractStringBuilder.java:730) [rt.jar:1.7.0_71]
>         at java.lang.StringBuilder.appendCodePoint(StringBuilder.java:242) [rt.jar:1.7.0_71]
>         at org.jboss.dmr.ModelValue.jsonEscape(ModelValue.java:212) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.BytesModelValue.formatAsJSON(BytesModelValue.java:144) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelNode.formatAsJSON(ModelNode.java:1520) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ObjectModelValue.formatAsJSON(ObjectModelValue.java:247) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelNode.formatAsJSON(ModelNode.java:1520) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ListModelValue.formatAsJSON(ListModelValue.java:251) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelNode.formatAsJSON(ModelNode.java:1520) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ObjectModelValue.formatAsJSON(ObjectModelValue.java:247) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelNode.formatAsJSON(ModelNode.java:1520) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ListModelValue.formatAsJSON(ListModelValue.java:251) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelNode.formatAsJSON(ModelNode.java:1520) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ObjectModelValue.formatAsJSON(ObjectModelValue.java:247) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelNode.formatAsJSON(ModelNode.java:1520) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ListModelValue.formatAsJSON(ListModelValue.java:251) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelNode.formatAsJSON(ModelNode.java:1520) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ObjectModelValue.formatAsJSON(ObjectModelValue.java:247) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelValue.writeJSONString(ModelValue.java:351) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelValue.toJSONString(ModelValue.java:340) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.dmr.ModelNode.toJSONString(ModelNode.java:1350) [jboss-dmr-1.2.0.Final-redhat-1.jar:1.2.0.Final-redhat-1]
>         at org.jboss.as.controller.audit.JsonAuditLogItemFormatter.createRecordText(JsonAuditLogItemFormatter.java:135) [jboss-as-controller-7.4.2.Final-redhat-2.jar:7.4.2.Final-redhat-2]
> A work-around is increasing the controller heap size, but it would be good if that was not required. If including the deployment content in the record is desired, perhaps there is a way to make it stream the record rather than building the entire string in memory at once.



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)