[keycloak-user] Keycloak OAuth High CPU usage

Tue May 24 02:31:20 EDT 2016

On 23 May 2016 at 10:02, Vaibhav Naldurgkar <
vaibhav_naldurgkar at persistent.com> wrote:

> Yes, the direct access grant is ON for this client. I am trying to
> understand what you mean by “not planning on using web based flow?” Could
> you provide more clarification on this.
>

If you are planning to do the web based flow (authorization code grant
flow) you should test with that rather than direct grant. That being said
the direct grant should still perform as well.

>
>
> This is what the scenario I am trying to execute and still have high CPU
> usages for KeyCloak Java process.
>
>
>
> ·         The end point URL
> /auth/realms/master/protocol/openid-connect/token has been called by Jmeter
> for 20 concurrent users per seconds to generate the tokens.
>
> ·         Even if used with crul command like “*curl -X POST -d
> "=admin&password=admin&password&client_id=HelloTest&grant_type=password"
> http://localhost:8080/auth/realms/master/protocol/openid-connect/token
> <http://localhost:8080/auth/realms/master/protocol/openid-connect/token>*”
> , in this case also the CPU utilizations goes around 100%.
>
> ·          After around 3 seconds of the test, in the output of top
> command on the KeyCloak server the CPU% for keycloak java process goes
> beyond 100%.
>
>
>
> Would it be possible for you to have a quick call for faster fix of this
> issue. This performance issue is holding to move KeyCloak to use as OAuth
> provider. If any other way is convenient for you please let me know for
> further discussion.
>

Your JMeter test is using 20 concurrent threads to send as many requests to
the direct grant api as it can. This will obviously cause Keycloak to
consume a high percentage of the CPU. Especially if you are running
everything on localhost as the network isn't going to be a bottleneck.
Neither will the database as Keycloak caches everything in memory. The
bottleneck will be the CPU.

Authenticating users and obtaining a token requires password hashing as
well as signing tokens, both are mainly CPU intensive. As you are using the
direct grant api there's also less network traffic.

You need to add some reports to your JMeter test so you can see how many
requests Keycloak can handle. That way you can find out how many users can
be authenticated per-second on your machine.

If you only have 500 users remember they won't all login at the same time
(seconds). Even if they all login at 9am sharp they will be spread out over
10 minutes or so, which would only be 1.2 logins/second.

>
>
> Thanks, Vaibhav
>
>
>
>
>
>
>
>
>
> *From:* Stian Thorgersen [mailto:sthorger at redhat.com]
> *Sent:* Monday, May 23, 2016 12:01 PM
>
> *To:* Vaibhav Naldurgkar
> *Cc:* keycloak-user at lists.jboss.org
> *Subject:* Re: [keycloak-user] Keycloak OAuth High CPU usage
>
>
>
> You are using direct grant to authenticate a user and obtain a token in
> the example above. This authenticates and creates a new session for each
> request. Are you not planning on using web based flow?
>
>
>
> What do you have password hashing intervals set to? Verifying password is
> CPU intensive, more than signing tokens.
>
>
>
> It shouldn't matter that user is stored in RedHat IdM as the user would be
> cached in Keycloak after first authentication, but it may be an idea to
> just double check by trying to authenticate to a user in Keycloak and not
> RH IdM.
>
>
>
> What results are you actually getting?
>
>
>
>
>
>
>
> On 20 May 2016 at 11:27, Vaibhav Naldurgkar <
> vaibhav_naldurgkar at persistent.com> wrote:
>
> Hi Stian,
>
>
>
> After reading your tests results of 10000 token refreshes in under 60
> seconds on your laptop, I am sure I am not following correct configuration
> and the documents are missing for reference.
>
>
>
> Could you please verify the below steps along with the screen-shots for
> the steps which I am following for the adding client and testing the Load
> performance using Jmeter. Please suggest if any changes are needed in the
> client configuration. In this case we are obtaining the token for user from
> KeyCloak.
>
>
>
> In my case the user have been stored on RedHat IdM which has been
> federated using KeyCloak.
>
>
>
>
>
> Step 1. Create new client called “LoadTest” , use the Client Protocol as
> “Openid-connect”.
>
> Used all defaults values post save of the client action.
>
>
>
> Step 2. Start the load tests using Jmeter and using the path as
> *“/auth/realms/master/protocol/openid-connect/token”* . Used 20 Number of
> Threads and used Post method.
>
>
>
>
>
> Below is the screen-shot for the step 1 related to Add Client.
>
>
>
>
>
>
>
>
>
> Below is the screen shot for the load test using Jmeter. In this case the
> Client ID was used as HelloTest.
>
>
>
>
>
>
>
> Http requests.
>
>
>
>
>
>
>
> Thanks, Vaibhav
>
>
>
>
>
> *From:* Stian Thorgersen [mailto:sthorger at redhat.com]
> *Sent:* Friday, May 20, 2016 1:01 PM
>
>
> *To:* Vaibhav Naldurgkar
> *Cc:* keycloak-user at lists.jboss.org
> *Subject:* Re: [keycloak-user] Keycloak OAuth High CPU usage
>
>
>
> Can you please elaborate a bit more on how your are testing scenario is?
> I'm a bit confused to what you are testing when you are talking about
> generating new tokens. Are you using OIDC or SAML? Are you talking about
> code->token exchanges, refresh token requests, or what?
>
>
>
> To test if your hardware is capable to deal with the load you need to test
> logins (verifying passwords are CPU intensive) as well as obtaining tokens
> (both code->token, done after login, and refreshing token, done ~1 min or
> so by active users, but most users won't continuously use the application).
>
>
>
> 500 users should be no problem at all. As an example with a single thread
> (which will use a single core) I could invoke 10000 token refreshes in
> under 60 seconds on my laptop. So a single core on my laptop should be able
> to handle 500 users.
>
>
>
> On 20 May 2016 at 08:00, Vaibhav Naldurgkar <
> vaibhav_naldurgkar at persistent.com> wrote:
>
> Hi Stian,
>
> Thank you for your reply.
>
>
>
> The new tokens needs to be generated for each user, which is needed from
> security point of view. The performance tests were also conducted using
> single Admin user and token for admin user; however in that case the
> performance was not good. In between 15th to 20th admin token access
> requests – the CPU usage of keycloak Java process was crossing 90 to 120%
> mark.
>
>
>
>
>
> As you have mentioned, Creating tokes are expected to be a bit CPU
> intensive – what should be the server configuration in terms of CPU to deal
> with more than 500 users to use keycloak as OAuth provider.
>
>
>
>
>
> Thanks, Vaibhav
>
>
>
>
>
>
>
> *From:* Stian Thorgersen [mailto:sthorger at redhat.com]
> *Sent:* Thursday, May 19, 2016 6:28 PM
> *To:* Vaibhav Naldurgkar
> *Cc:* keycloak-user at lists.jboss.org
> *Subject:* Re: [keycloak-user] Keycloak OAuth High CPU usage
>
>
>
> Creating tokes are expected to be a bit CPU intensive as they need to be
> signed. When you say you try to generate tokens for 10-20 users are you
> doing performance tests and having 10-20 threads generating tokens? It
> shouldn't make any difference if you have 10 or if you have 200 users, it's
> the total number of tokens that can be generated that's an issue. Having
> 200 concurrent users with a access token timeout of 60 seconds should mean
> that you need to be able to generate roughly 200/60 tokens = 3.3 tokens/sec.
>
>
>
> On 19 May 2016 at 13:24, Vaibhav Naldurgkar <
> vaibhav_naldurgkar at persistent.com> wrote:
>
> Hi All,
>
>
>
> I am using Keycloak 1.9.3 with default configuration. Keycloak server is
> installed on RHEL 6.5 virtual image with 4 CPU , 8 GB RAM and java version
> is jdk1.8.0_73 We are trying to use keycloak as a OAuth provider. But when
> we try and generate token(
> http:///auth/realms/master/protocol/openid-connect/token
> <http://auth/realms/master/protocol/openid-connect/token>) for more than
> 10-20 users the server gets too slow and cpu usage goes over 100%.
>
> Any pointers on how to improve performance of keycloak OAuth provider. We
> need to support at least 200 concurrent users.
>
>
>
>
>
> Thanks, Vaibhav
>
> DISCLAIMER ========== This e-mail may contain privileged and confidential
> information which is the property of Persistent Systems Ltd. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of
> this message. Persistent Systems Ltd. does not accept any liability for
> virus infected mails.
>
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
>
>
> DISCLAIMER ========== This e-mail may contain privileged and confidential
> information which is the property of Persistent Systems Ltd. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of
> this message. Persistent Systems Ltd. does not accept any liability for
> virus infected mails.
>
>
>
> DISCLAIMER ========== This e-mail may contain privileged and confidential
> information which is the property of Persistent Systems Ltd. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of
> this message. Persistent Systems Ltd. does not accept any liability for
> virus infected mails.
>
>
>
> DISCLAIMER ========== This e-mail may contain privileged and confidential
> information which is the property of Persistent Systems Ltd. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of
> this message. Persistent Systems Ltd. does not accept any liability for
> virus infected mails.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20160524/b46c50e2/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 101405 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/keycloak-user/attachments/20160524/b46c50e2/attachment-0003.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 11865 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/keycloak-user/attachments/20160524/b46c50e2/attachment-0004.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 18447 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/keycloak-user/attachments/20160524/b46c50e2/attachment-0005.png 