[Red Hat JIRA] (WFLY-14434) Heap usage continuously growing and exhausting available heap memory in production
by Paul Ferraro (Jira)
[ https://issues.redhat.com/browse/WFLY-14434?page=com.atlassian.jira.plugi... ]
Paul Ferraro commented on WFLY-14434:
-------------------------------------
[~manas.java] Is this issue reproducible on the latest release, i.e. 22.0.1.Final?
> Heap usage continuously growing and exhausting available heap memory in production
> ----------------------------------------------------------------------------------
>
> Key: WFLY-14434
> URL: https://issues.redhat.com/browse/WFLY-14434
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Reporter: Manas Panda
> Assignee: Paul Ferraro
> Priority: Major
> Attachments: Heap_Size_keeps_increasing.jpg, Heap_dump_analysis_1.jpg
>
>
> # Problem description:
> Critical issue of heap usage continuously growing and exhausting available heap memory in production. As you can see in below heap dump org.infinispan.container.impl.DefaultDataContainer is growing as the time passes and not garbage collected by GC. This increase in heap happening after upgrading from Wildfly 10.1 to Wildfly 18.0.1. There is no change in JDK in both cases ( Wildfly 10.1 and Wildfly 18.0.1).
> 2. Web application environment
> * OS: RHEL 7.5
> * Wildfly 18.0.1
> * JDK 1.8
> * Wildfly is being run in cluster mode
> * Integrated with Keycloak 3.4.3 for SSO ( SAML)
> * Enabled Wildfly clustering mode
> * G1GC garbage collector used. And 20gigs of heap allocated ( -Xmx)
> * Environment of Web App on Wildfly 10.1 is same ( except Wildfly 18) with same hardware
> * Web application uses Spring web framework and security
> 3. Heap dump analysis
> * We have few web users logging in and every second and external application consuming few API’s exposed by same application.
> * As you can see infinispan.container.impl.DefaultDataContainer Has already grown 14gigs.
> * Please note that web application does not use infinispan directly.
> * The heap does not grow in wildfly 10.1 and its normal ( garbage gets collected and size gets reduced after GC)
> Please find the snapshot as attached.
--
This message was sent by Atlassian Jira
(v8.13.1#813001)
5 years, 2 months
[Red Hat JIRA] (WFCORE-4827) Errors Missing on Invalid Configuration
by Brian Stansberry (Jira)
[ https://issues.redhat.com/browse/WFCORE-4827?page=com.atlassian.jira.plug... ]
Brian Stansberry commented on WFCORE-4827:
------------------------------------------
Thanks. So AIUI the issue is that steps that add services result in adding a step the executes a ServiceVerificationHelper that monitors those particular services. And the first ServiceVerificationHelper that fails prevents any later ServiceVerificationHelper steps running, because the first one triggers rollback.
One solution to this is to change this:
https://github.com/wildfly/wildfly-core/blob/master/controller/src/main/j...
Instead of immediately calling context.setRollbackOnly, instead add another Stage.VERIFY step that calls it. That step will be added to the end of the queue for Stage.VERIFY, so it will execute after all the previously registered ServiceVerificationHelper steps.
> Errors Missing on Invalid Configuration
> ---------------------------------------
>
> Key: WFCORE-4827
> URL: https://issues.redhat.com/browse/WFCORE-4827
> Project: WildFly Core
> Issue Type: Bug
> Components: Security
> Affects Versions: 11.0.0.Beta7
> Reporter: Darran Lofthouse
> Assignee: Richard Opalka
> Priority: Critical
> Labels: domain-mode
>
> [~ropalka] I believe this is caused by the MSC refactoring.
> Steps, in the default host.xml for domain mode.
> 1. Define the following security realm: -
> {noformat}
> <security-realms>
> <security-realm name="ldap_security_realm">
> <server-identities>
> <ssl>
> <keystore path="generated.keystore" relative-to="jboss.server.config.dir" keystore-password="password" alias="server" key-password="password" generate-self-signed-certificate-host="localhost"/>
> </ssl>
> </server-identities>
> <authentication>
> <ldap connection="testLdap" base-dn="dc=test,dc=sbc,dc=com" recursive="true">
> <username-filter attribute="samaccountname"/>
> </ldap>
> </authentication>
> </security-realm>
> {noformat}
> 2. Define the following outbound connection: -
> {noformat}
> <outbound-connections>
> <ldap name="testLdap" url="ldap://localhost:636" search-dn="CN=mxxxxxx,OU=GenericID,OU=testUsers,DC=testServices,DC=test,DC=com" search-credential="passowrd" />
> </outbound-connections>
> {noformat}
> 3. Update the management interfaces to: -
> {noformat}
> <management-interfaces>
> <http-interface security-realm="ldap_security_realm">
> <http-upgrade enabled="true"/>
> <socket interface="management" port="${jboss.management.http.port:9990}"/>
> </http-interface>
> </management-interfaces>
> {noformat}
> The server fails to boot with just the following error: -
> {noformat}
> [Host Controller] 17:56:40,052 FATAL [org.jboss.as.host.controller] (Controller Boot Thread) WFLYHC0034: Host Controller boot has failed in an unrecoverable manner; exiting. See previous messages for details.
> {noformat}
> If the management interface is then updated to reference the ManagementRealm instead the error is now: -
> {noformat}
> [Host Controller] 18:01:48,595 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([
> [Host Controller] ("host" => "master"),
> [Host Controller] ("core-service" => "management"),
> [Host Controller] ("security-realm" => "ldap_security_realm")
> [Host Controller] ]) - failure description: {
> [Host Controller] "WFLYCTL0412: Required services that are not installed:" => ["jboss.server.path.\"jboss.server.config.dir\""],
> [Host Controller] "WFLYCTL0180: Services with missing/unavailable dependencies" => ["org.wildfly.core.management.security.realm.ldap_security_realm.key-manager is missing [jboss.server.path.\"jboss.server.config.dir\"]"]
> [Host Controller] }
> {noformat}
> This error is expected as the realm defined in step 1 referenced an invalid path.
> I believe the error reporting should come from this method: -
> org.jboss.as.controller.ServiceVerificationHelper.execute(OperationContext, ModelNode)
> However something seems to have changes with the MSC migration.
> This was recently encountered debugging the bug report in https://issues.redhat.com/browse/WFCORE-4820, if you see an error "Multiple CallbackHandlerServices for the same mechanism (PLAIN)" that has been covered by WFCORE-4820.
--
This message was sent by Atlassian Jira
(v8.13.1#813001)
5 years, 2 months
[Red Hat JIRA] (WFLY-14434) Heap usage continuously growing and exhausting available heap memory in production
by Brian Stansberry (Jira)
[ https://issues.redhat.com/browse/WFLY-14434?page=com.atlassian.jira.plugi... ]
Brian Stansberry reassigned WFLY-14434:
---------------------------------------
Component/s: Clustering
Assignee: Paul Ferraro (was: Brian Stansberry)
Issue Type: Bug (was: Task)
Priority: Major (was: Blocker)
> Heap usage continuously growing and exhausting available heap memory in production
> ----------------------------------------------------------------------------------
>
> Key: WFLY-14434
> URL: https://issues.redhat.com/browse/WFLY-14434
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Reporter: Manas Panda
> Assignee: Paul Ferraro
> Priority: Major
> Attachments: Heap_Size_keeps_increasing.jpg, Heap_dump_analysis_1.jpg
>
>
> # Problem description:
> Critical issue of heap usage continuously growing and exhausting available heap memory in production. As you can see in below heap dump org.infinispan.container.impl.DefaultDataContainer is growing as the time passes and not garbage collected by GC. This increase in heap happening after upgrading from Wildfly 10.1 to Wildfly 18.0.1. There is no change in JDK in both cases ( Wildfly 10.1 and Wildfly 18.0.1).
> 2. Web application environment
> * OS: RHEL 7.5
> * Wildfly 18.0.1
> * JDK 1.8
> * Wildfly is being run in cluster mode
> * Integrated with Keycloak 3.4.3 for SSO ( SAML)
> * Enabled Wildfly clustering mode
> * G1GC garbage collector used. And 20gigs of heap allocated ( -Xmx)
> * Environment of Web App on Wildfly 10.1 is same ( except Wildfly 18) with same hardware
> * Web application uses Spring web framework and security
> 3. Heap dump analysis
> * We have few web users logging in and every second and external application consuming few API’s exposed by same application.
> * As you can see infinispan.container.impl.DefaultDataContainer Has already grown 14gigs.
> * Please note that web application does not use infinispan directly.
> * The heap does not grow in wildfly 10.1 and its normal ( garbage gets collected and size gets reduced after GC)
> Please find the snapshot as attached.
--
This message was sent by Atlassian Jira
(v8.13.1#813001)
5 years, 2 months
[Red Hat JIRA] (WFLY-14433) WildFly 22. Compile JSP error. JBWEB004062
by Brian Stansberry (Jira)
[ https://issues.redhat.com/browse/WFLY-14433?page=com.atlassian.jira.plugi... ]
Brian Stansberry reassigned WFLY-14433:
---------------------------------------
Component/s: Web (Undertow)
Assignee: Flavia Rainone (was: Brian Stansberry)
> WildFly 22. Compile JSP error. JBWEB004062
> ------------------------------------------
>
> Key: WFLY-14433
> URL: https://issues.redhat.com/browse/WFLY-14433
> Project: WildFly
> Issue Type: Bug
> Components: Web (Undertow)
> Affects Versions: 22.0.1.Final
> Reporter: Jose Gracia
> Assignee: Flavia Rainone
> Priority: Major
>
> Hello,
> When I try to run an application on WildFly 22.0.1 server, I get the following error:
> ---------------------------------------------------------
> Stacktrace:
> INFO JBWEB004062: Unable to compile class for JSP:
> JBWEB004060: An error occurred at line: 12 in the jsp file: /jsps/global/error.jsp
> Syntax error on tokens, delete these tokens
> 9: <%
> 10: // Introducing the exception in the User container
> 11: UserContainer userContainer = (UserContainer)session.getAttribute(StrutsKeys.SES_USER_CONTAINER);
> 12: if(exception instanceof java.net.SocketException){
> 13: }else{
> 14: if (userContainer == null) {
> 15: userContainer = new UserContainer();
> ---------------------------------------------------------
> Best regards.
--
This message was sent by Atlassian Jira
(v8.13.1#813001)
5 years, 2 months