[jboss-jira] [JBoss JIRA] (ELY-1687) Initial WildFly Elytron Performance Enhancments

[ https://issues.jboss.org/browse/ELY-1687?page=com.atlassian.jira.plugin.s... ] Darran Lofthouse updated ELY-1687: ---------------------------------- Description: Rather than this becoming a single long running task to review the performance of WildFly Elytron I think the best strategy is to identity a test strategy, obtain some metrics of that strategy under load, perform profiling to identity a set of issues and look at options to address those issues. After that we will perform the initial metric test again and close the issue. A new issue will then be created either to repeat the same test or start with a new test which may be a subtle change of the first test. The first test is to test HTTP Basic authentication backed by WildFly Elytron. * Each client will alternatively send a request with no authorization header so triggering a HTTP 401 challenge followed by a request including the header which should successfully authenticate. Attached is a JMeter test plan configured to use 250 client threads, each submitting requests for 5 minutes. h2. Initial Issues h3. WildFlyElytronProvider Locking Total block time 8.393s via calls to java.security.Provider.getServices(); Potentially something that could be eliminated if mechanisms were loaded in advance, or at the very least the factories were loaded in advance. h3. Memory 2.42G of char[] e.g. {noformat} void java.nio.HeapCharBuffer.<init>(int, int) 13037 CharBuffer java.nio.CharBuffer.allocate(int) 9148 CharBuffer java.nio.charset.CharsetDecoder.decode(ByteBuffer) 9148 CharBuffer java.nio.charset.Charset.decode(ByteBuffer) 9148 void org.wildfly.security.http.impl.BasicAuthenticationMechanism.evaluateRequest(HttpServerRequest) 9148 {noformat} Is there any option to re-use these as they can be cleared instead of leaving to GC. HeapByteBuffer and HeapCharBuffer are also quite prominent. h3. Memory 1.78G of Callback[] Using the CallbackHandler API the use of the array is inevitable. * Could we extend the API to avoid the array? * Could we re-use the array? Could consider null termination. {noformat} boolean org.wildfly.security.http.impl.UsernamePasswordAuthenticationMechanism.authenticate(String, String, char[]) 9222 void org.wildfly.security.http.impl.BasicAuthenticationMechanism.evaluateRequest(HttpServerRequest) 9222 {noformat} h3. Memory 1.41G of HttpAuthenticator$Builder {noformat} HttpAuthenticator$Builder org.wildfly.security.http.HttpAuthenticator.builder() 24699 boolean org.wildfly.elytron.web.undertow.server.SecurityContextImpl.authenticate() 24699 {noformat} Could we switch to something that associated these with a ThreadLocal and update the API to allow re-use? h3. Memory 1.3G of SecurityContextImpl {noformat} SecurityContext org.wildfly.elytron.web.undertow.server.SecurityContextImpl$Builder.build() 3247 SecurityContext org.wildfly.elytron.web.undertow.server.ElytronContextAssociationHandler.createSecurityContext(HttpServerExchange) 1673 {noformat} Also instances of HttpAuthenticator {noformat} HttpAuthenticator org.wildfly.security.http.HttpAuthenticator$Builder.build() 14624 {noformat} And instances of HttpAuthenticator$AuthenticationExchange. {noformat} boolean org.wildfly.security.http.HttpAuthenticator.authenticate() 14423 {noformat} As with HttpAuthenticator$Builder is there any way to consider re-use? h3. Memory 1.21G of java.util.ArrayList {noformat} boolean org.wildfly.security.http.HttpAuthenticator$AuthenticationExchange.authenticate() 8911 {noformat} Can check the use and see if an alternative is possible. _If this mechanism was re-written as a recursive call it would eliminate the need for the intermediate ArrayList to hold the responders._ _This will still be a worthwhile improvement but may need to keep in mind this ArrayList size likely includes the responders which means it includes the mechs and the additional references._ https://issues.jboss.org/browse/ELYWEB-26 h3. Memory ServerAuthenticationContext States Each ServerAuthenticationContext State is it's own class which needs to be instantiated, a single authentication requests results in multiple states. Should the state machine be internal to the ServerAuthenticationContext so we have only one class instance? h3. Memory 885Mb of Undertow FormParserFactory$ParserDefinition[] {noformat} FormParserFactory$Builder io.undertow.server.handlers.form.FormParserFactory.builder(boolean) 1091 FormParserFactory$Builder io.undertow.server.handlers.form.FormParserFactory.builder() 1091 void org.wildfly.elytron.web.undertow.server.ElytronHttpExchange.<init>(HttpServerExchange, Map, ScopeSessionListener) 1091 {noformat} This test did not use forms at all, is this something that can be delayed until we know it is needed? _It may be possible for the FormParserFactory to be a single static reference, the parser it self is created on a per-request basis as needed._ https://issues.jboss.org/browse/ELYWEB-27 h3. Memory SecurityIdentity As an immutable object we can end up with intermediate throw away instances, can we optimise create once? {noformat} SecurityIdentity org.wildfly.security.auth.server.SecurityIdentity.withPrivateCredentials(IdentityCredentials) 11454 ServerAuthenticationContext$AuthorizedAuthenticationState org.wildfly.security.auth.server.ServerAuthenticationContext$NameAssignedState.doAuthorization(boolean) 11454 {noformat} h3. Method Profiling - org.wildfly.common.iteration.ByteArrayIterator and ByteIterator These lead to multiple instances of different classes, and the iteration is flagging in the top 10 packages. Could a static Base64 conversion clean up a lot of this? h1. Done h3. Method Profiling - new HttpString A lot of time spend creating new HttpString (Package is no3 in the top list) {noformat} void io.undertow.util.HttpString.<init>(String) 4 void org.wildfly.elytron.web.undertow.server.ElytronHttpExchange.addResponseHeader(String, String) 4 {noformat} Could we re-use the HttpString for common header types? Re-use of HttpString from cache https://issues.jboss.org/browse/ELYWEB-25 was: Rather than this becoming a single long running task to review the performance of WildFly Elytron I think the best strategy is to identity a test strategy, obtain some metrics of that strategy under load, perform profiling to identity a set of issues and look at options to address those issues. After that we will perform the initial metric test again and close the issue. A new issue will then be created either to repeat the same test or start with a new test which may be a subtle change of the first test. The first test is to test HTTP Basic authentication backed by WildFly Elytron. * Each client will alternatively send a request with no authorization header so triggering a HTTP 401 challenge followed by a request including the header which should successfully authenticate. Attached is a JMeter test plan configured to use 250 client threads, each submitting requests for 5 minutes. h2. Initial Issues h3. WildFlyElytronProvider Locking Total block time 8.393s via calls to java.security.Provider.getServices(); Potentially something that could be eliminated if mechanisms were loaded in advance, or at the very least the factories were loaded in advance. h3. Memory 2.42G of char[] e.g. {noformat} void java.nio.HeapCharBuffer.<init>(int, int) 13037 CharBuffer java.nio.CharBuffer.allocate(int) 9148 CharBuffer java.nio.charset.CharsetDecoder.decode(ByteBuffer) 9148 CharBuffer java.nio.charset.Charset.decode(ByteBuffer) 9148 void org.wildfly.security.http.impl.BasicAuthenticationMechanism.evaluateRequest(HttpServerRequest) 9148 {noformat} Is there any option to re-use these as they can be cleared instead of leaving to GC. HeapByteBuffer and HeapCharBuffer are also quite prominent. h3. Memory 1.78G of Callback[] Using the CallbackHandler API the use of the array is inevitable. * Could we extend the API to avoid the array? * Could we re-use the array? Could consider null termination. {noformat} boolean org.wildfly.security.http.impl.UsernamePasswordAuthenticationMechanism.authenticate(String, String, char[]) 9222 void org.wildfly.security.http.impl.BasicAuthenticationMechanism.evaluateRequest(HttpServerRequest) 9222 {noformat} h3. Memory 1.41G of HttpAuthenticator$Builder {noformat} HttpAuthenticator$Builder org.wildfly.security.http.HttpAuthenticator.builder() 24699 boolean org.wildfly.elytron.web.undertow.server.SecurityContextImpl.authenticate() 24699 {noformat} Could we switch to something that associated these with a ThreadLocal and update the API to allow re-use? h3. Memory 1.3G of SecurityContextImpl {noformat} SecurityContext org.wildfly.elytron.web.undertow.server.SecurityContextImpl$Builder.build() 3247 SecurityContext org.wildfly.elytron.web.undertow.server.ElytronContextAssociationHandler.createSecurityContext(HttpServerExchange) 1673 {noformat} Also instances of HttpAuthenticator {noformat} HttpAuthenticator org.wildfly.security.http.HttpAuthenticator$Builder.build() 14624 {noformat} And instances of HttpAuthenticator$AuthenticationExchange. {noformat} boolean org.wildfly.security.http.HttpAuthenticator.authenticate() 14423 {noformat} As with HttpAuthenticator$Builder is there any way to consider re-use? h3. Memory 1.21G of java.util.ArrayList {noformat} boolean org.wildfly.security.http.HttpAuthenticator$AuthenticationExchange.authenticate() 8911 {noformat} Can check the use and see if an alternative is possible. _If this mechanism was re-written as a recursive call it would eliminate the need for the intermediate ArrayList to hold the responders._ _This will still be a worthwhile improvement but may need to keep in mind this ArrayList size likely includes the responders which means it includes the mechs and the additional references._ https://issues.jboss.org/browse/ELYWEB-26 h3. Memory ServerAuthenticationContext States Each ServerAuthenticationContext State is it's own class which needs to be instantiated, a single authentication requests results in multiple states. Should the state machine be internal to the ServerAuthenticationContext so we have only one class instance? h3. Memory 885Mb of Undertow FormParserFactory$ParserDefinition[] {noformat} FormParserFactory$Builder io.undertow.server.handlers.form.FormParserFactory.builder(boolean) 1091 FormParserFactory$Builder io.undertow.server.handlers.form.FormParserFactory.builder() 1091 void org.wildfly.elytron.web.undertow.server.ElytronHttpExchange.<init>(HttpServerExchange, Map, ScopeSessionListener) 1091 {noformat} This test did not use forms at all, is this something that can be delayed until we know it is needed? _It may be possible for the FormParserFactory to be a single static reference, the parser it self is created on a per-request basis as needed._ h3. Memory SecurityIdentity As an immutable object we can end up with intermediate throw away instances, can we optimise create once? {noformat} SecurityIdentity org.wildfly.security.auth.server.SecurityIdentity.withPrivateCredentials(IdentityCredentials) 11454 ServerAuthenticationContext$AuthorizedAuthenticationState org.wildfly.security.auth.server.ServerAuthenticationContext$NameAssignedState.doAuthorization(boolean) 11454 {noformat} h3. Method Profiling - org.wildfly.common.iteration.ByteArrayIterator and ByteIterator These lead to multiple instances of different classes, and the iteration is flagging in the top 10 packages. Could a static Base64 conversion clean up a lot of this? h1. Done h3. Method Profiling - new HttpString A lot of time spend creating new HttpString (Package is no3 in the top list) {noformat} void io.undertow.util.HttpString.<init>(String) 4 void org.wildfly.elytron.web.undertow.server.ElytronHttpExchange.addResponseHeader(String, String) 4 {noformat} Could we re-use the HttpString for common header types? Re-use of HttpString from cache https://issues.jboss.org/browse/ELYWEB-25 -- This message was sent by Atlassian Jira (v7.12.1#712002)