[JBoss JIRA] (WFCORE-1560) Cli calls leak resources in Host Controller when repeatedly calling jboss-cli.sh
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-1560?page=com.atlassian.jira.plugi... ]
Brian Stansberry updated WFCORE-1560:
-------------------------------------
Component/s: Domain Management
(was: CLI)
> Cli calls leak resources in Host Controller when repeatedly calling jboss-cli.sh
> --------------------------------------------------------------------------------
>
> Key: WFCORE-1560
> URL: https://issues.jboss.org/browse/WFCORE-1560
> Project: WildFly Core
> Issue Type: Bug
> Components: Domain Management
> Affects Versions: 2.0.8.Final
> Environment: OS: CentOS 7.2
> Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)
> Wildfly-10.0.0-Final
> Reporter: Michael Noack
> Assignee: Alexey Loubyansky
> Attachments: console-dc.log, host-controller.log, JVM-DC.png, process-controller.log
>
>
> When executing management commands using jboss-cli.sh against the domain controller of a cluster repeatedly the host controller uses up more and more memory in oldgen. After several thousands of runs of jboss-cli the host controller eventually becomes unresponsive (see attached picture for memory consumption, dc became entirely unresponsive at roughly 6:30am):
> [root@dc broken]# /opt/wildfly-10.0.0.Final-DC/bin/./jboss-cli.sh --connect --user="username" --password="password" --command=":read-children-names(child-type=host)"
> Failed to connect to the controller: The controller is not available at xx.xx.xx.xx:9993: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://xx.xx.xx.xx:9993. The connection timed out: WFLYPRT0023: Could not connect to https-remoting://xx.xx.xx.xx:9993. The connection timed out
> I discovered the issue when testing whether https://issues.jboss.org/browse/WFCORE-974 was actually resolved in wildfly-10.0.0.Final as advertised. I can confirm that the issue is different, since no OOM-Exceptions are thrown. However the DC still becomes useless, since it won't accept any connections anymore. -I will check whether the work-around from WFCORE-974 applies to this issue as well.- However the work-around from WFCORE-974 doesn't fix this issue.
> Please note that the attached logs are UTC, while the monitoring is UTC+2. Also the collection values are misleading since I haven't adapted my monitoring to the new output of jstat in JDK8. PU and PC are thus MU and MC.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-1560) Cli calls leak resources in Host Controller when repeatedly calling jboss-cli.sh
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-1560?page=com.atlassian.jira.plugi... ]
Brian Stansberry reassigned WFCORE-1560:
----------------------------------------
Assignee: Brian Stansberry (was: Alexey Loubyansky)
> Cli calls leak resources in Host Controller when repeatedly calling jboss-cli.sh
> --------------------------------------------------------------------------------
>
> Key: WFCORE-1560
> URL: https://issues.jboss.org/browse/WFCORE-1560
> Project: WildFly Core
> Issue Type: Bug
> Components: Domain Management
> Affects Versions: 2.0.8.Final
> Environment: OS: CentOS 7.2
> Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)
> Wildfly-10.0.0-Final
> Reporter: Michael Noack
> Assignee: Brian Stansberry
> Attachments: console-dc.log, host-controller.log, JVM-DC.png, process-controller.log
>
>
> When executing management commands using jboss-cli.sh against the domain controller of a cluster repeatedly the host controller uses up more and more memory in oldgen. After several thousands of runs of jboss-cli the host controller eventually becomes unresponsive (see attached picture for memory consumption, dc became entirely unresponsive at roughly 6:30am):
> [root@dc broken]# /opt/wildfly-10.0.0.Final-DC/bin/./jboss-cli.sh --connect --user="username" --password="password" --command=":read-children-names(child-type=host)"
> Failed to connect to the controller: The controller is not available at xx.xx.xx.xx:9993: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://xx.xx.xx.xx:9993. The connection timed out: WFLYPRT0023: Could not connect to https-remoting://xx.xx.xx.xx:9993. The connection timed out
> I discovered the issue when testing whether https://issues.jboss.org/browse/WFCORE-974 was actually resolved in wildfly-10.0.0.Final as advertised. I can confirm that the issue is different, since no OOM-Exceptions are thrown. However the DC still becomes useless, since it won't accept any connections anymore. -I will check whether the work-around from WFCORE-974 applies to this issue as well.- However the work-around from WFCORE-974 doesn't fix this issue.
> Please note that the attached logs are UTC, while the monitoring is UTC+2. Also the collection values are misleading since I haven't adapted my monitoring to the new output of jstat in JDK8. PU and PC are thus MU and MC.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-730) Expose MANIFEST.MF deployments metadata in DMR
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-730?page=com.atlassian.jira.plugin... ]
Brian Stansberry reassigned WFCORE-730:
---------------------------------------
Assignee: ehsavoie Hugonnet (was: Jason Greene)
Emmanuel: another one the :read-content will handle.
> Expose MANIFEST.MF deployments metadata in DMR
> -----------------------------------------------
>
> Key: WFCORE-730
> URL: https://issues.jboss.org/browse/WFCORE-730
> Project: WildFly Core
> Issue Type: Feature Request
> Components: Domain Management
> Reporter: Thomas Heute
> Assignee: ehsavoie Hugonnet
>
> This is a request to expose metadata contained in META-INF/MANIFEST.MF though DMR at least for deployments.
> For management we wish we could read the version of a deployed artifact if this was filled-in in the WAR/EAR(/JAR?)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-730) Expose MANIFEST.MF deployments metadata in DMR
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-730?page=com.atlassian.jira.plugin... ]
Brian Stansberry commented on WFCORE-730:
-----------------------------------------
Note that we intend to expose deployment content via the WildFly core management API, not "via DMR". DMR is just a data format library used in message exchange. It's not a name for WildFly management in general. We don't intend to convert deployment content into DMR ModelNodes.
> Expose MANIFEST.MF deployments metadata in DMR
> -----------------------------------------------
>
> Key: WFCORE-730
> URL: https://issues.jboss.org/browse/WFCORE-730
> Project: WildFly Core
> Issue Type: Feature Request
> Components: Domain Management
> Reporter: Thomas Heute
> Assignee: Jason Greene
>
> This is a request to expose metadata contained in META-INF/MANIFEST.MF though DMR at least for deployments.
> For management we wish we could read the version of a deployed artifact if this was filled-in in the WAR/EAR(/JAR?)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-601) Boot sometimes fails on restart when deployments are present in the deployments directory and the deployment scanner is in use
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-601?page=com.atlassian.jira.plugin... ]
Brian Stansberry updated WFCORE-601:
------------------------------------
Component/s: Deployment Scanner
(was: Server)
> Boot sometimes fails on restart when deployments are present in the deployments directory and the deployment scanner is in use
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: WFCORE-601
> URL: https://issues.jboss.org/browse/WFCORE-601
> Project: WildFly Core
> Issue Type: Bug
> Components: Deployment Scanner
> Reporter: Stuart Douglas
> Assignee: ehsavoie Hugonnet
>
> I observed this today, looks to be some kind of race in the deployment scanner on boot
> {code}
> 21:52:08,618 INFO [org.jboss.modules] (main) JBoss Modules version 1.4.1.Final
> 21:52:10,370 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.4.Final
> 21:52:10,440 INFO [org.jboss.as] (MSC service thread 1-6) WFLYSRV0049: WildFly Full 9.0.0.Alpha2-SNAPSHOT (WildFly Core 1.0.0.Alpha19) starting
> 21:52:11,390 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 20) WFLYCTL0028: Attribute enabled is deprecated, and it might be removed in future version!
> 21:52:11,409 INFO [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0015: Re-attempting failed deployment mysql-ds.xml
> 21:52:11,413 INFO [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0015: Re-attempting failed deployment wildfly-ee7.war
> 21:52:11,488 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 3) WFLYDR0001: Content added at location /Users/stuart/workspace/wildfly/build/target/wildfly-9.0.0.Alpha2-SNAPSHOT/standalone/data/content/48/032d14f289775249a004205c059588ab0f44d4/content
> 21:52:11,504 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 3) WFLYDR0001: Content added at location /Users/stuart/workspace/wildfly/build/target/wildfly-9.0.0.Alpha2-SNAPSHOT/standalone/data/content/c6/7babde4791697ddc6509fbd43ba1ec25952d47/content
> 21:52:11,506 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 3) WFLYCTL0013: Operation ("add") failed - address: ([("deployment" => "wildfly-ee7.war")]) - failure description: "WFLYCTL0212: Duplicate resource [(\"deployment\" => \"wildfly-ee7.war\")]"
> 21:52:11,509 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) "WFLYCTL0193: Failed executing subsystem deployment-scanner boot operations"
> 21:52:11,512 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("parallel-subsystem-boot") failed - address: ([]) - failure description: "\"WFLYCTL0193: Failed executing subsystem deployment-scanner boot operations\""
> 21:52:11,514 FATAL [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0056: Server boot has failed in an unrecoverable manner; exiting. See previous messages for details.
> 21:52:11,514 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-456) Add a content-stream attribute to the root deployment=* resources
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-456?page=com.atlassian.jira.plugin... ]
Brian Stansberry reassigned WFCORE-456:
---------------------------------------
Assignee: ehsavoie Hugonnet
Emmanuel, this is another one that I think your :read-content work covers, so I'm assigning it to you to resolve as out of date or whatever once you've got the :read-content stuff done.
> Add a content-stream attribute to the root deployment=* resources
> -----------------------------------------------------------------
>
> Key: WFCORE-456
> URL: https://issues.jboss.org/browse/WFCORE-456
> Project: WildFly Core
> Issue Type: Feature Request
> Components: Domain Management
> Reporter: Brian Stansberry
> Assignee: ehsavoie Hugonnet
>
> Add a new attribute to the deployment resource that allows the user to get a stream with the deployment content attached to the response, a la what we do with the subsystem=logging/log-file=* resource's "stream" attribute.
> The attribute must be configured as runtime-only and read-only.
> The read handler that handles the attribute should only attach a stream if the following conditions are met, otherwise the value for the attribute should remain as 'undefined'.
> 1) The resource's "content" attribute has only a single element in the list. (This is always the case today; adding a check for this is just future proofing)
> 2) The first element in the "content" attribute list has a defined value in its "hash" field. (This confirms the content is stored in the content repo. We won't support serving unmanaged content as we have insufficient control over it.)
> 3) The first element in the "content" attribute list has a 'true' as value of its "archive" field. (Current this will always be the case; this is future proofing for when we add support for exploded content in the content repo. Trying to assemble a stream from exploded content is currently out of scope for this feature.)
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFLY-6858) read-resource should return the job file name
by James Perkins (JIRA)
[ https://issues.jboss.org/browse/WFLY-6858?page=com.atlassian.jira.plugin.... ]
James Perkins commented on WFLY-6858:
-------------------------------------
If it helps too we could have a list of all the valid XML files on the base {{/deployment=*/subsystem=batch-jberet}} resource too.
{code}
{
"outcome" => "success",
"result" => {
"job-xml-files" => [
"simple.xml",
"retry-chunk.xml",
"partition-chunk.xml"
],
"job" => {
"simple" => {
"instance-count" => 0,
"job-xml-files" => ["simple.xml"],
"running-executions" => 0,
"execution" => undefined
},
"chunkPartition" => {
"instance-count" => 0,
"job-xml-files" => [
"retry-chunk.xml",
"partition-chunk.xml"
],
"running-executions" => 0,
"execution" => undefined
}
}
}
}
{code}
> read-resource should return the job file name
> ---------------------------------------------
>
> Key: WFLY-6858
> URL: https://issues.jboss.org/browse/WFLY-6858
> Project: WildFly
> Issue Type: Bug
> Components: Batch
> Reporter: Claudio Miranda
> Assignee: James Perkins
>
> The start-job operation must pass the job file name as parameter,
> /deployment=*/subsystem=batch-jberet:read-operation-description(name=start-job)
> however the jon attribute below, returns the job-id name, not the job file name
> /deployment=*/subsystem=batch-jberet/job=*
> So there is no way to know at runtime, the job file name. Is it possible to return the job xml file name ?
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFLY-6858) read-resource should return the job file name
by James Perkins (JIRA)
[ https://issues.jboss.org/browse/WFLY-6858?page=com.atlassian.jira.plugin.... ]
James Perkins commented on WFLY-6858:
-------------------------------------
[~claudio4j] Would something like the following work?
{code}
{
"outcome" => "success",
"result" => {"job" => {
"simple" => {
"instance-count" => 0,
"job-xml-files" => ["simple.xml"],
"running-executions" => 0,
"execution" => undefined
},
"chunkPartition" => {
"instance-count" => 0,
"job-xml-files" => [
"retry-chunk.xml",
"partition-chunk.xml"
],
"running-executions" => 0,
"execution" => undefined
}
}}
}
{code}
The new attribute is {{job-xml-files}}.
> read-resource should return the job file name
> ---------------------------------------------
>
> Key: WFLY-6858
> URL: https://issues.jboss.org/browse/WFLY-6858
> Project: WildFly
> Issue Type: Bug
> Components: Batch
> Reporter: Claudio Miranda
> Assignee: James Perkins
>
> The start-job operation must pass the job file name as parameter,
> /deployment=*/subsystem=batch-jberet:read-operation-description(name=start-job)
> however the jon attribute below, returns the job-id name, not the job file name
> /deployment=*/subsystem=batch-jberet/job=*
> So there is no way to know at runtime, the job file name. Is it possible to return the job xml file name ?
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months
[JBoss JIRA] (WFCORE-345) Provide way to display deployed application metadata
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-345?page=com.atlassian.jira.plugin... ]
Brian Stansberry commented on WFCORE-345:
-----------------------------------------
Thanks; that was my interpretation too. :) Sorry for the slightly tardy question. I'll move this to WFLY and the Web - Undertow component as I'm pretty sure if we did anything on it, it would be exposed in the management tree under /deployment=foo.war/subsystem=undertow.
> Provide way to display deployed application metadata
> ----------------------------------------------------
>
> Key: WFCORE-345
> URL: https://issues.jboss.org/browse/WFCORE-345
> Project: WildFly Core
> Issue Type: Feature Request
> Components: Domain Management
> Reporter: Nicklas Karlsson
> Assignee: Tomaz Cerar
>
> It would be handy to have the deployment process print out the complete DeploymentInfo for an application so one could view the final order of filters and request listeners etc.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years, 8 months