[keycloak-dev] JMeter Performance test

Tue Jul 1 05:42:01 EDT 2014

On 1.7.2014 10:51, Stian Thorgersen wrote:
> Merged
>
> One quick question can you easily point the performance tests to an external server? So that we can test with the WildFly dist rather than embedded Undertow.
Yes, you can point it to any server you want. There are just 2 things 
though:

1) Performance test is using different users right now (IMO much more 
close to "real" environment than using same username in all sessions), 
so it needs users to be created before running jmeter test. So 
KeycloakPerfServer (embedded undertow) now uses perf-tools and users are 
created with it. So the Wildfly/AS7 would also need perf-tools deployed 
and users created with that.

2) It uses simple web application for performance testing. So the 
application needs to be deployed as well.

I was also thinking about using Keycloak TokenService endpoints 
directly,however in the web application there is some pre/post 
processing and parsing of tokens etc, which is much easier to do in Java 
then in JMeter itself. which allows to check for example that correct 
user is logged and so it's easier to verify that performance test is 
doing what is is supposed to do and using different user in each session 
etc.

Do you think it makes sense to have also test for TokenService endpoints 
directly? Was also wondering about adding tests for AccountService as 
this might be used by any user, so this stuff can be used quite often? 
Not sure about admin console as this won't be hardly used by many users 
at the same time (load won't be so big to admin console endpoints).

Marek
>
> ----- Original Message -----
>> From: "Marek Posolda" <mposolda at redhat.com>
>> To: keycloak-dev at lists.jboss.org
>> Sent: Monday, 30 June, 2014 9:08:48 AM
>> Subject: [keycloak-dev] JMeter Performance test
>>
>> I've sent PR https://github.com/keycloak/keycloak/pull/490 with first
>> prototype of Keycloak performance test with JMeter. For now it's using
>> embedded Undertow (reused KeycloakServer stuff from testsuite) also with
>> performance-tools, so it's easy to add new users. Most of the stuff is
>> configurable through System properties, which allows to easily switch
>> different databases and models without any needed changes in
>> configuration files (For example PostgreSQL, MySQL, Mongo,
>> enabled/disabled simple cache etc), which is also good for easy setup of
>> performance test in Jenkins to have it in CI. Some details in README:
>> testsuite/performance-web/README.md .
>>
>> If you like the approach, I can continue and possibly setup CI. For now,
>> I've tried some testing on my laptop with 10.000 users in database and
>> each user having 1 realm role (available through scope to tested web
>> application) and 0 application roles... (I guess having configurable
>> number of roles, scopes per user might be useful as well and I can
>> possibly add it? )
>>
>> Each test is using 50 clients (each client having different username, so
>> using users like user-0, user-1, user-49 ) and each client doing 50 test
>> iterations. Each iteration is doing:
>> - Redirect to KC login screen
>> - Login into KC
>> - Exchange code for token
>> - Refresh token 2 times
>> - Logout
>>
>> Some results from test on my laptop:
>>
>> PostgreSQL with simple cache disabled:
>>
>> uri                                       count     total     min
>> average max
>> "Login request"                  2500    381151    10    152 1168
>> "Confirm Login request"    2500    1202384    39    480    1505
>> "Exchange code request"  2500    1162489    28    464    1677
>> "RefreshToken request"     5000    1992961    22    398    1630
>> "Logout request"               2500    786237    21    314 1383
>>
>>
>> PostgreSQL with simple cache enabled:
>>
>> uri                                      count     total     min average
>> max
>> "Login request"                  2500    168819    6    67    857
>> "Confirm Login request"    2500    412464    18    164    1205
>> "Exchange code request"  2500    670047    21    268    1121
>> "RefreshToken request"     5000    997389    10    199    976
>> "Logout request"               2500    475672    17    190 1328
>>
>>
>> Mongo model with simple cache disabled:
>>
>> uri                                       count     total     min
>> average max
>> "Login request"                  2500    171976    6    68 1014
>> "Confirm Login request"    2500    412258    11    164    981
>> "Exchange code request"  2500    512125    12    204    1144
>> "RefreshToken request"     5000    923467    11    184    1073
>> "Logout request"               2500    367173    10    146 1108
>>
>>
>> Mongo model with simple cache enabled:
>>
>> uri                                      count     total     min average
>> max
>> "Login request"                  2500    135489    5    54 1085
>> "Confirm Login request"    2500    224540    7    89    1318
>> "Exchange code request"  2500    343649    9    137    905
>> "RefreshToken request"     5000    556202    8    111    750
>> "Logout request"               2500    292986    10    117 1574
>>
>> Marek
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>