[jboss-jira] [JBoss JIRA] (WFCORE-1560) Cli calls leak resources in Host Controller when repeatedly calling jboss-cli.sh

[Ken Wills (JIRA)]

    [ https://issues.jboss.org/browse/WFCORE-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397849#comment-13397849 ] 

Ken Wills edited comment on WFCORE-1560 at 4/25/17 11:39 AM:
-------------------------------------------------------------

After a lot of invocations and watching the heap usage, I don't think this is still an issue. There are two exceptions that occur very infrequently that cause the client to abort (the server process is still fine when the client starts again), that I'm collecting info on and will open separate tickets for these.

The heap size eventually comes back up to about 320M, and used oscillates from 72M->236M.


was (Author: luck3y):
After a lot of invocations and watching the heap usage, I don't think this is still an issue. There are two exceptions that occur very infrequently that cause the client to abort (the server process is still fine when the client starts again), that I'm collecting info on and will open separate tickets for these.

> Cli calls leak resources in Host Controller when repeatedly calling jboss-cli.sh
> --------------------------------------------------------------------------------
>
>                 Key: WFCORE-1560
>                 URL: https://issues.jboss.org/browse/WFCORE-1560
>             Project: WildFly Core
>          Issue Type: Bug
>          Components: Domain Management
>    Affects Versions: 2.0.8.Final
>         Environment: OS: CentOS 7.2
> Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)
> Wildfly-10.0.0-Final
>            Reporter: Michael Noack
>            Assignee: Ken Wills
>            Priority: Critical
>             Fix For: 3.0.0.Beta17
>
>         Attachments: JVM-DC.png, console-dc.log, host-controller.log, process-controller.log
>
>
> When executing management commands using jboss-cli.sh against the domain controller of a cluster repeatedly the host controller uses up more and more memory in oldgen. After several thousands of runs of jboss-cli the host controller eventually becomes unresponsive (see attached picture for memory consumption, dc became entirely unresponsive at roughly 6:30am):
> [root at dc broken]# /opt/wildfly-10.0.0.Final-DC/bin/./jboss-cli.sh --connect  --user="username" --password="password" --command=":read-children-names(child-type=host)"
> Failed to connect to the controller: The controller is not available at xx.xx.xx.xx:9993: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://xx.xx.xx.xx:9993. The connection timed out: WFLYPRT0023: Could not connect to https-remoting://xx.xx.xx.xx:9993. The connection timed out
> I discovered the issue when testing whether https://issues.jboss.org/browse/WFCORE-974 was actually resolved in wildfly-10.0.0.Final as advertised. I can confirm that the issue is different, since no OOM-Exceptions are thrown. However the DC still becomes useless, since it won't accept any connections anymore. -I will check whether the work-around from WFCORE-974 applies to this issue as well.- However the work-around from WFCORE-974 doesn't fix this issue.
> Please note that the attached logs are UTC, while the monitoring is UTC+2. Also the collection values are misleading since I haven't adapted my monitoring to the new output of jstat in JDK8. PU and PC are thus MU and MC.



--
This message was sent by Atlassian JIRA
(v7.2.3#72005)