Initial WildFly Elytron Performance Enhancments ----------------------------------------------- Key: ELY-1687 URL: https://issues.redhat.com/browse/ELY-1687 Project: WildFly Elytron Issue Type: Task Affects Versions: 1.7.0.CR2 Reporter: Darran Lofthouse Assignee: Darran Lofthouse Priority: Major Fix For: 1.12.0.CR2 Attachments: BASIC_Auth_Load.jmx, Flight.tgz Rather than this becoming a single long running task to review the performance of WildFly Elytron I think the best strategy is to identity a test strategy, obtain some metrics of that strategy under load, perform profiling to identity a set of issues and look at options to address those issues. After that we will perform the initial metric test again and close the issue. A new issue will then be created either to repeat the same test or start with a new test which may be a subtle change of the first test. The first test is to test HTTP Basic authentication backed by WildFly Elytron. * Each client will alternatively send a request with no authorization header so triggering a HTTP 401 challenge followed by a request including the header which should successfully authenticate. Attached is a JMeter test plan configured to use 250 client threads, each submitting requests for 5 minutes. h2. Initial Issues h3. WildFlyElytronProvider Locking Total block time 8.393s via calls to java.security.Provider.getServices(); Potentially something that could be eliminated if mechanisms were loaded in advance, or at the very least the factories were loaded in advance. h3. Memory 2.42G of char[] e.g. {noformat} void java.nio.HeapCharBuffer.<init>(int, int) 13037 CharBuffer java.nio.CharBuffer.allocate(int) 9148 CharBuffer java.nio.charset.CharsetDecoder.decode(ByteBuffer) 9148 CharBuffer java.nio.charset.Charset.decode(ByteBuffer) 9148 void org.wildfly.security.http.impl.BasicAuthenticationMechanism.evaluateRequest(HttpServerRequest) 9148 {noformat} Is there any option to re-use these as they can be cleared instead of leaving to GC. HeapByteBuffer and HeapCharBuffer are also quite prominent. h3. Memory 1.78G of Callback[] Using the CallbackHandler API the use of the array is inevitable. * Could we extend the API to avoid the array? * Could we re-use the array? Could consider null termination. {noformat} boolean org.wildfly.security.http.impl.UsernamePasswordAuthenticationMechanism.authenticate(String, String, char[]) 9222 void org.wildfly.security.http.impl.BasicAuthenticationMechanism.evaluateRequest(HttpServerRequest) 9222 {noformat} h3. Memory 1.41G of HttpAuthenticator$Builder {noformat} HttpAuthenticator$Builder org.wildfly.security.http.HttpAuthenticator.builder() 24699 boolean org.wildfly.elytron.web.undertow.server.SecurityContextImpl.authenticate() 24699 {noformat} Could we switch to something that associated these with a ThreadLocal and update the API to allow re-use? h3. Memory 1.3G of SecurityContextImpl {noformat} SecurityContext org.wildfly.elytron.web.undertow.server.SecurityContextImpl$Builder.build() 3247 SecurityContext org.wildfly.elytron.web.undertow.server.ElytronContextAssociationHandler.createSecurityContext(HttpServerExchange) 1673 {noformat} Also instances of HttpAuthenticator {noformat} HttpAuthenticator org.wildfly.security.http.HttpAuthenticator$Builder.build() 14624 {noformat} And instances of HttpAuthenticator$AuthenticationExchange. {noformat} boolean org.wildfly.security.http.HttpAuthenticator.authenticate() 14423 {noformat} As with HttpAuthenticator$Builder is there any way to consider re-use? h3. Memory 1.21G of java.util.ArrayList {noformat} boolean org.wildfly.security.http.HttpAuthenticator$AuthenticationExchange.authenticate() 8911 {noformat} Can check the use and see if an alternative is possible. _If this mechanism was re-written as a recursive call it would eliminate the need for the intermediate ArrayList to hold the responders._ _This will still be a worthwhile improvement but may need to keep in mind this ArrayList size likely includes the responders which means it includes the mechs and the additional references._ https://issues.jboss.org/browse/ELY-1689 h3. Memory ServerAuthenticationContext States Each ServerAuthenticationContext State is it's own class which needs to be instantiated, a single authentication requests results in multiple states. Should the state machine be internal to the ServerAuthenticationContext so we have only one class instance? h3. Memory SecurityIdentity As an immutable object we can end up with intermediate throw away instances, can we optimise create once? {noformat} SecurityIdentity org.wildfly.security.auth.server.SecurityIdentity.withPrivateCredentials(IdentityCredentials) 11454 ServerAuthenticationContext$AuthorizedAuthenticationState org.wildfly.security.auth.server.ServerAuthenticationContext$NameAssignedState.doAuthorization(boolean) 11454 {noformat} h3. Method Profiling - org.wildfly.common.iteration.ByteArrayIterator and ByteIterator These lead to multiple instances of different classes, and the iteration is flagging in the top 10 packages. Could a static Base64 conversion clean up a lot of this? h1. Done h3. Method Profiling - new HttpString A lot of time spend creating new HttpString (Package is no3 in the top list) {noformat} void io.undertow.util.HttpString.<init>(String) 4 void org.wildfly.elytron.web.undertow.server.ElytronHttpExchange.addResponseHeader(String, String) 4 {noformat} Could we re-use the HttpString for common header types? Re-use of HttpString from cache https://issues.jboss.org/browse/ELYWEB-25 h3. Memory 885Mb of Undertow FormParserFactory$ParserDefinition[] {noformat} FormParserFactory$Builder io.undertow.server.handlers.form.FormParserFactory.builder(boolean) 1091 FormParserFactory$Builder io.undertow.server.handlers.form.FormParserFactory.builder() 1091 void org.wildfly.elytron.web.undertow.server.ElytronHttpExchange.<init>(HttpServerExchange, Map, ScopeSessionListener) 1091 {noformat} This test did not use forms at all, is this something that can be delayed until we know it is needed? _It may be possible for the FormParserFactory to be a single static reference, the parser it self is created on a per-request basis as needed._ https://issues.jboss.org/browse/ELYWEB-27