May 2020 - jboss-jira - Jboss List Archives

[JBoss JIRA] (ELY-1687) Initial WildFly Elytron Performance Enhancments

by Darran Lofthouse (Jira)

[ https://issues.redhat.com/browse/ELY-1687?page=com.atlassian.jira.plugin.... ] Darran Lofthouse updated ELY-1687: ---------------------------------- Fix Version/s: (was: 1.12.0.CR2) > Initial WildFly Elytron Performance Enhancments > ----------------------------------------------- > > Key: ELY-1687 > URL: https://issues.redhat.com/browse/ELY-1687 > Project: WildFly Elytron > Issue Type: Task > Affects Versions: 1.7.0.CR2 > Reporter: Darran Lofthouse > Assignee: Darran Lofthouse > Priority: Major > Attachments: BASIC_Auth_Load.jmx, Flight.tgz > > > Rather than this becoming a single long running task to review the performance of WildFly Elytron I think the best strategy is to identity a test strategy, obtain some metrics of that strategy under load, perform profiling to identity a set of issues and look at options to address those issues. > After that we will perform the initial metric test again and close the issue. > A new issue will then be created either to repeat the same test or start with a new test which may be a subtle change of the first test. > The first test is to test HTTP Basic authentication backed by WildFly Elytron. > * Each client will alternatively send a request with no authorization header so triggering a HTTP 401 challenge followed by a request including the header which should successfully authenticate. > Attached is a JMeter test plan configured to use 250 client threads, each submitting requests for 5 minutes. > h2. Initial Issues > h3. WildFlyElytronProvider Locking > Total block time 8.393s via calls to java.security.Provider.getServices(); > Potentially something that could be eliminated if mechanisms were loaded in advance, or at the very least the factories were loaded in advance. > h3. Memory 2.42G of char[] > e.g. > {noformat} > void java.nio.HeapCharBuffer.<init>(int, int) 13037 > CharBuffer java.nio.CharBuffer.allocate(int) 9148 > CharBuffer java.nio.charset.CharsetDecoder.decode(ByteBuffer) 9148 > CharBuffer java.nio.charset.Charset.decode(ByteBuffer) 9148 > void org.wildfly.security.http.impl.BasicAuthenticationMechanism.evaluateRequest(HttpServerRequest) 9148 > {noformat} > Is there any option to re-use these as they can be cleared instead of leaving to GC. > HeapByteBuffer and HeapCharBuffer are also quite prominent. > h3. Memory 1.78G of Callback[] > Using the CallbackHandler API the use of the array is inevitable. > * Could we extend the API to avoid the array? > * Could we re-use the array? Could consider null termination. > {noformat} > boolean org.wildfly.security.http.impl.UsernamePasswordAuthenticationMechanism.authenticate(String, String, char[]) 9222 > void org.wildfly.security.http.impl.BasicAuthenticationMechanism.evaluateRequest(HttpServerRequest) 9222 > {noformat} > h3. Memory 1.41G of HttpAuthenticator$Builder > {noformat} > HttpAuthenticator$Builder org.wildfly.security.http.HttpAuthenticator.builder() 24699 > boolean org.wildfly.elytron.web.undertow.server.SecurityContextImpl.authenticate() 24699 > {noformat} > Could we switch to something that associated these with a ThreadLocal and update the API to allow re-use? > h3. Memory 1.3G of SecurityContextImpl > {noformat} > SecurityContext org.wildfly.elytron.web.undertow.server.SecurityContextImpl$Builder.build() 3247 > SecurityContext org.wildfly.elytron.web.undertow.server.ElytronContextAssociationHandler.createSecurityContext(HttpServerExchange) 1673 > {noformat} > Also instances of HttpAuthenticator > {noformat} > HttpAuthenticator org.wildfly.security.http.HttpAuthenticator$Builder.build() 14624 > {noformat} > And instances of HttpAuthenticator$AuthenticationExchange. > {noformat} > boolean org.wildfly.security.http.HttpAuthenticator.authenticate() 14423 > {noformat} > As with HttpAuthenticator$Builder is there any way to consider re-use? > h3. Memory 1.21G of java.util.ArrayList > {noformat} > boolean org.wildfly.security.http.HttpAuthenticator$AuthenticationExchange.authenticate() 8911 > {noformat} > Can check the use and see if an alternative is possible. > _If this mechanism was re-written as a recursive call it would eliminate the need for the intermediate ArrayList to hold the responders._ > _This will still be a worthwhile improvement but may need to keep in mind this ArrayList size likely includes the responders which means it includes the mechs and the additional references._ > https://issues.jboss.org/browse/ELY-1689 > h3. Memory ServerAuthenticationContext States > Each ServerAuthenticationContext State is it's own class which needs to be instantiated, a single authentication requests results in multiple states. > Should the state machine be internal to the ServerAuthenticationContext so we have only one class instance? > h3. Memory SecurityIdentity > As an immutable object we can end up with intermediate throw away instances, can we optimise create once? > {noformat} > SecurityIdentity org.wildfly.security.auth.server.SecurityIdentity.withPrivateCredentials(IdentityCredentials) 11454 > ServerAuthenticationContext$AuthorizedAuthenticationState org.wildfly.security.auth.server.ServerAuthenticationContext$NameAssignedState.doAuthorization(boolean) 11454 > {noformat} > h3. Method Profiling - org.wildfly.common.iteration.ByteArrayIterator and ByteIterator > These lead to multiple instances of different classes, and the iteration is flagging in the top 10 packages. > Could a static Base64 conversion clean up a lot of this? > h1. Done > h3. Method Profiling - new HttpString > A lot of time spend creating new HttpString (Package is no3 in the top list) > {noformat} > void io.undertow.util.HttpString.<init>(String) 4 > void org.wildfly.elytron.web.undertow.server.ElytronHttpExchange.addResponseHeader(String, String) 4 > {noformat} > Could we re-use the HttpString for common header types? > Re-use of HttpString from cache https://issues.jboss.org/browse/ELYWEB-25 > h3. Memory 885Mb of Undertow FormParserFactory$ParserDefinition[] > {noformat} > FormParserFactory$Builder io.undertow.server.handlers.form.FormParserFactory.builder(boolean) 1091 > FormParserFactory$Builder io.undertow.server.handlers.form.FormParserFactory.builder() 1091 > void org.wildfly.elytron.web.undertow.server.ElytronHttpExchange.<init>(HttpServerExchange, Map, ScopeSessionListener) 1091 > {noformat} > This test did not use forms at all, is this something that can be delayed until we know it is needed? > _It may be possible for the FormParserFactory to be a single static reference, the parser it self is created on a per-request basis as needed._ > https://issues.jboss.org/browse/ELYWEB-27 -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (WFCORE-4967) RequestController drops queued tasks randomly during server start.

by Darran Lofthouse (Jira)

[ https://issues.redhat.com/browse/WFCORE-4967?page=com.atlassian.jira.plug... ] Darran Lofthouse commented on WFCORE-4967: ------------------------------------------ By RequestController I assume you mean the class "org.wildfly.extension.requestcontroller.RequestController" ? > RequestController drops queued tasks randomly during server start. > ------------------------------------------------------------------ > > Key: WFCORE-4967 > URL: https://issues.redhat.com/browse/WFCORE-4967 > Project: WildFly Core > Issue Type: Bug > Components: Server > Affects Versions: 11.0.0.Final > Reporter: P K > Assignee: Jeff Mesnil > Priority: Major > > Our deployment is quite complex, EAR app containing several modules. > After move to different data-center some of EJB timers will not be triggered at all after server restart (randomly). > There was no change to wildfly server version and application itself. > Only hardware/network changed and issue started occurring. > Classified as : WEIRDO and some race conditions was assumed. -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (WFCORE-4967) RequestController drops queued tasks randomly during server start.

by P K (Jira)

[ https://issues.redhat.com/browse/WFCORE-4967?page=com.atlassian.jira.plug... ] P K updated WFCORE-4967: ------------------------ Steps to Reproduce: I created a minimal example project which demonstrates this EJB timer issue: https://github.com/guestlinelabs/wildfly-timer-issue h2. Recreation steps # Deploy sample application to the wildfly server (via cli or just by copy it to standalone/deployments). # Start the server h5. Note Above order is very important because when app is deployed after RequestController is resumed, issue is not occurring. h3. Expected All timers from SomeTimers class are triggered after the server is resumed. This can be confirmed by finding an entry in server log at INFO level, which message starts with timers OK h3. Incorrect behaviour Some (random) timers are omitted, there is a message in logs saying timers FAIL or there is no log entry at all. was:I will create minimal deployment and provide a link here as I know what is the cause already. > RequestController drops queued tasks randomly during server start. > ------------------------------------------------------------------ > > Key: WFCORE-4967 > URL: https://issues.redhat.com/browse/WFCORE-4967 > Project: WildFly Core > Issue Type: Bug > Components: Server > Affects Versions: 11.0.0.Final > Reporter: P K > Assignee: Jeff Mesnil > Priority: Major > > Our deployment is quite complex, EAR app containing several modules. > After move to different data-center some of EJB timers will not be triggered at all after server restart (randomly). > There was no change to wildfly server version and application itself. > Only hardware/network changed and issue started occurring. > Classified as : WEIRDO and some race conditions was assumed. -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (WFCORE-4967) RequestController drops queued tasks randomly during server start.

by P K (Jira)

[ https://issues.redhat.com/browse/WFCORE-4967?page=com.atlassian.jira.plug... ] P K commented on WFCORE-4967: ----------------------------- Identified Cause: RequestController is not thread safe when in paused mode. •Entry: ControlPoint.requestComplete() used by ManagedExecutorServiceImpl / ManagedScheduledExecutorImpl •problematic method: RequestController.findForcedTask() > RequestController drops queued tasks randomly during server start. > ------------------------------------------------------------------ > > Key: WFCORE-4967 > URL: https://issues.redhat.com/browse/WFCORE-4967 > Project: WildFly Core > Issue Type: Bug > Components: Server > Affects Versions: 11.0.0.Final > Reporter: P K > Assignee: Jeff Mesnil > Priority: Major > > Our deployment is quite complex, EAR app containing several modules. > After move to different data-center some of EJB timers will not be triggered at all after server restart (randomly). > There was no change to wildfly server version and application itself. > Only hardware/network changed and issue started occurring. > Classified as : WEIRDO and some race conditions was assumed. -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (WFCORE-4967) RequestController drops queued tasks randomly during server start.

by P K (Jira)

[ https://issues.redhat.com/browse/WFCORE-4967?page=com.atlassian.jira.plug... ] P K updated WFCORE-4967: ------------------------ Description: Our deployment is quite complex, EAR app containing several modules. After move to different data-center some of EJB timers will not be triggered at all after server restart (randomly). There was no change to wildfly server version and application itself. Only hardware/network changed and issue started occurring. Classified as : WEIRDO and some race conditions was assumed. was: Our deployment is quite complex, EAR app containing several modules. After move to different data-center some of EJB timers will not be triggered at all after server restart (randomly). There was no change to wildfly server version and application itself. Only hardware/network changed and issue started occurring. Classified as : WEIRDO and some race conditions was assumed. > RequestController drops queued tasks randomly during server start. > ------------------------------------------------------------------ > > Key: WFCORE-4967 > URL: https://issues.redhat.com/browse/WFCORE-4967 > Project: WildFly Core > Issue Type: Bug > Components: Server > Affects Versions: 11.0.0.Final > Reporter: P K > Assignee: Jeff Mesnil > Priority: Major > > Our deployment is quite complex, EAR app containing several modules. > After move to different data-center some of EJB timers will not be triggered at all after server restart (randomly). > There was no change to wildfly server version and application itself. > Only hardware/network changed and issue started occurring. > Classified as : WEIRDO and some race conditions was assumed. -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (WFCORE-4967) RequestController drops queued tasks randomly during server start.

by P K (Jira)

P K created WFCORE-4967: --------------------------- Summary: RequestController drops queued tasks randomly during server start. Key: WFCORE-4967 URL: https://issues.redhat.com/browse/WFCORE-4967 Project: WildFly Core Issue Type: Bug Components: Server Affects Versions: 11.0.0.Final Reporter: P K Assignee: Jeff Mesnil Our deployment is quite complex, EAR app containing several modules. After move to different data-center some of EJB timers will not be triggered at all after server restart (randomly). There was no change to wildfly server version and application itself. Only hardware/network changed and issue started occurring. Classified as : WEIRDO and some race conditions was assumed. -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (SWSQE-1147) Fix the failed calling webhook error in install-istio

by Filip Brychta (Jira)

Filip Brychta created SWSQE-1147: ------------------------------------ Summary: Fix the failed calling webhook error in install-istio Key: SWSQE-1147 URL: https://issues.redhat.com/browse/SWSQE-1147 Project: Kiali QE Issue Type: QE Task Reporter: Filip Brychta Assignee: Filip Brychta Sometimes the install-istio fails with: Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "validation.istio.io": Post https://istiod.istio-system.svc:443/validate?timeout=30s: x509: certificate has expired or is not yet valid (repeated 1 times) -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (SWSQE-1126) Update kiali-build jenkins slave to use operator-sdk 0.17.0

by Filip Brychta (Jira)

[ https://issues.redhat.com/browse/SWSQE-1126?page=com.atlassian.jira.plugi... ] Filip Brychta resolved SWSQE-1126. ---------------------------------- Resolution: Done > Update kiali-build jenkins slave to use operator-sdk 0.17.0 > ----------------------------------------------------------- > > Key: SWSQE-1126 > URL: https://issues.redhat.com/browse/SWSQE-1126 > Project: Kiali QE > Issue Type: QE Task > Reporter: Filip Brychta > Assignee: Filip Brychta > Priority: Major > Labels: infrastructure > -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (SWSQE-1137) Build Kiali-Operator PRs

by Filip Brychta (Jira)

[ https://issues.redhat.com/browse/SWSQE-1137?page=com.atlassian.jira.plugi... ] Filip Brychta closed SWSQE-1137. -------------------------------- Resolution: Duplicate Dup of SWSQE-1119 > Build Kiali-Operator PRs > ------------------------ > > Key: SWSQE-1137 > URL: https://issues.redhat.com/browse/SWSQE-1137 > Project: Kiali QE > Issue Type: QE Task > Reporter: Matthew Mahoney > Assignee: Filip Brychta > Priority: Major > > In addition to being able to build Kiali by Kiali and Kiali-UI PRs, we now will need to build by Kiali-Operator PRs given that the operator is in it's own repo. -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (SWSQE-996) Jenkins Job For Disconnected OCP Configurtation

by Filip Brychta (Jira)

[ https://issues.redhat.com/browse/SWSQE-996?page=com.atlassian.jira.plugin... ] Filip Brychta closed SWSQE-996. ------------------------------- Resolution: Duplicate Dup of SWSQE-1132 > Jenkins Job For Disconnected OCP Configurtation > ----------------------------------------------- > > Key: SWSQE-996 > URL: https://issues.redhat.com/browse/SWSQE-996 > Project: Kiali QE > Issue Type: QE Task > Reporter: Matthew Mahoney > Priority: Critical > Labels: infrastructure > > There is a need to test in Disconnected Configurations. Create Jenkins job to build Disconnected 4.x clusters. > References: > * [Installing Restricted Network Preparations|https://docs.openshift.com/container-platform/4.2/installing...] > * [Disconnected Installation for Operators|https://docs.google.com/document/d/1n4MJgVk8g254s_X6oOgzC1z44Rc...] > * [Your Operator and Disconnected Clusters|https://docs.google.com/document/d/1P4vsuf_yQ8Pl0Fpd1ZQMbHEL2bvt...] -- This message was sent by Atlassian Jira (v7.13.8#713008)

6 years, 2 months

1
0
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

jboss-jira May 2020