[keycloak-dev] JMeter Performance test

[Marek Posolda]

I've sent PR https://github.com/keycloak/keycloak/pull/490 with first 
prototype of Keycloak performance test with JMeter. For now it's using 
embedded Undertow (reused KeycloakServer stuff from testsuite) also with 
performance-tools, so it's easy to add new users. Most of the stuff is 
configurable through System properties, which allows to easily switch 
different databases and models without any needed changes in 
configuration files (For example PostgreSQL, MySQL, Mongo, 
enabled/disabled simple cache etc), which is also good for easy setup of 
performance test in Jenkins to have it in CI. Some details in README: 
testsuite/performance-web/README.md .

If you like the approach, I can continue and possibly setup CI. For now, 
I've tried some testing on my laptop with 10.000 users in database and 
each user having 1 realm role (available through scope to tested web 
application) and 0 application roles... (I guess having configurable 
number of roles, scopes per user might be useful as well and I can 
possibly add it? )

Each test is using 50 clients (each client having different username, so 
using users like user-0, user-1, user-49 ) and each client doing 50 test 
iterations. Each iteration is doing:
- Redirect to KC login screen
- Login into KC
- Exchange code for token
- Refresh token 2 times
- Logout

Some results from test on my laptop:

PostgreSQL with simple cache disabled:

uri                                       count     total     min     
average max
"Login request"                  2500    381151    10    152 1168
"Confirm Login request"    2500    1202384    39    480    1505
"Exchange code request"  2500    1162489    28    464    1677
"RefreshToken request"     5000    1992961    22    398    1630
"Logout request"               2500    786237    21    314 1383


PostgreSQL with simple cache enabled:

uri                                      count     total     min average 
max
"Login request"                  2500    168819    6    67    857
"Confirm Login request"    2500    412464    18    164    1205
"Exchange code request"  2500    670047    21    268    1121
"RefreshToken request"     5000    997389    10    199    976
"Logout request"               2500    475672    17    190 1328


Mongo model with simple cache disabled:

uri                                       count     total     min     
average max
"Login request"                  2500    171976    6    68 1014
"Confirm Login request"    2500    412258    11    164    981
"Exchange code request"  2500    512125    12    204    1144
"RefreshToken request"     5000    923467    11    184    1073
"Logout request"               2500    367173    10    146 1108


Mongo model with simple cache enabled:

uri                                      count     total     min average 
max
"Login request"                  2500    135489    5    54 1085
"Confirm Login request"    2500    224540    7    89    1318
"Exchange code request"  2500    343649    9    137    905
"RefreshToken request"     5000    556202    8    111    750
"Logout request"               2500    292986    10    117 1574

Marek