8:53 a.m.
Hi,
we're running nightly stress tests against our keycloak dev environment,
to monitor maximum throughput rates and average response times of
selected endpoints.
After upgrading from KC 3.4.3 to 4.8.3, we noticed considerable dents in
our curves.
For example:
Userinfo dropped from ~12k max. Requests per sec to ~7k; response times
increased from ~30ms to 45ms
Code flow (3 correlated requests): 1.5k -> 1k max. Requests per sec;
100ms -> 150ms response time
Password Credentials Grant: 800 -> 600 max. Requests per sec; 300ms ->
500ms response time
We have another system running KC 4.5.0, which does not seem to suffer
from that performance decrease, so it was probably introduced with >
4.5.
Are there any known developments that might be the reason for our
observations?
Thanks,
Mario.
10:09 a.m.
Hello Mario,
Thanks for letting us know. I will try and have a look into it this week and try to
isolate the cause of that regression.
Regards,
Tomas
----- Original Message -----
Hi,
we're running nightly stress tests against our keycloak dev environment,
to monitor maximum throughput rates and average response times of
selected endpoints.
After upgrading from KC 3.4.3 to 4.8.3, we noticed considerable dents in
our curves.
For example:
Userinfo dropped from ~12k max. Requests per sec to ~7k; response times
increased from ~30ms to 45ms
Code flow (3 correlated requests): 1.5k -> 1k max. Requests per sec;
100ms -> 150ms response time
Password Credentials Grant: 800 -> 600 max. Requests per sec; 300ms ->
500ms response time
We have another system running KC 4.5.0, which does not seem to suffer
from that performance decrease, so it was probably introduced with >
4.5.
Are there any known developments that might be the reason for our
observations?
Thanks,
Mario.
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-user
11:50 a.m.
Hi,
I tried to reproduce the regression between 4.5.0 and 4.8.3 by running a simple OIDC login
/ refresh token / logout scenario, but I din't see any regression neither in
single-node deployment or 2-node cluster deployment. I can see almost the same performance
numbers in throughput and response times (within 3% difference).
Is it possible that some of your configuration changed during the upgrade? For example
data-source settings (connection pool size, etc.) or max. number of HTTP connections on
the container's HTTP listener.
You can check by running: `./jboss-cli.sh -c` from your KC installation and then:
/subsystem=undertow/server=default-server/http-listener=default:read-resource
--> attribute max-connections
/subsystem=datasources/data-source=KeycloakDS:read-resource
--> attributes min-pool-size, max-pool-size or others which could affect performance
Perhaps some settings on the load-balancer if you have a clustered deployment? Some custom
SPI implementations?
One setting which has big performance impact is the number of hashing iterations. However
I checked and the default setting is the same in both versions: 27500.
Anyway, thanks for letting us know. We will be adding more test coverage into the
performance testsuite in the near future so we'll try to keep this in mind.
CC: Marek, Hynek - any ideas off the top of your head what could have changed between
4.5.0 and 4.8.3 which could have caused a regression in the mentioned endpoints?
Regards,
Tomas
----- Original Message -----
Hello Mario,
Thanks for letting us know. I will try and have a look into it this week and
try to isolate the cause of that regression.
Regards,
Tomas
----- Original Message -----
> Hi,
>
> we're running nightly stress tests against our keycloak dev environment,
> to monitor maximum throughput rates and average response times of
> selected endpoints.
>
> After upgrading from KC 3.4.3 to 4.8.3, we noticed considerable dents in
> our curves.
>
> For example:
> Userinfo dropped from ~12k max. Requests per sec to ~7k; response times
> increased from ~30ms to 45ms
> Code flow (3 correlated requests): 1.5k -> 1k max. Requests per sec;
> 100ms -> 150ms response time
> Password Credentials Grant: 800 -> 600 max. Requests per sec; 300ms ->
> 500ms response time
>
> We have another system running KC 4.5.0, which does not seem to suffer
> from that performance decrease, so it was probably introduced with >
> 4.5.
>
> Are there any known developments that might be the reason for our
> observations?
>
> Thanks,
> Mario.
> _______________________________________________
> keycloak-user mailing list
> keycloak-user(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-user
1:30 a.m.
On 07/03/2019 18:50, Tomas Kyjovsky wrote:
Hi,
I tried to reproduce the regression between 4.5.0 and 4.8.3 by running a simple OIDC
login / refresh token / logout scenario, but I din't see any regression neither in
single-node deployment or 2-node cluster deployment. I can see almost the same performance
numbers in throughput and response times (within 3% difference).
Is it possible that some of your configuration changed during the upgrade? For example
data-source settings (connection pool size, etc.) or max. number of HTTP connections on
the container's HTTP listener.
You can check by running: `./jboss-cli.sh -c` from your KC installation and then:
/subsystem=undertow/server=default-server/http-listener=default:read-resource
--> attribute max-connections
/subsystem=datasources/data-source=KeycloakDS:read-resource
--> attributes min-pool-size, max-pool-size or others which could affect
performance
Perhaps some settings on the load-balancer if you have a clustered deployment? Some
custom SPI implementations?
One setting which has big performance impact is the number of hashing iterations. However
I checked and the default setting is the same in both versions: 27500.
Anyway, thanks for letting us know. We will be adding more test coverage into the
performance testsuite in the near future so we'll try to keep this in mind.
CC: Marek, Hynek - any ideas off the top of your head what could have changed between
4.5.0 and 4.8.3 which could have caused a regression in the mentioned endpoints?
I can't recall anything from the top of my head, but it's possible there
were some changes affecting performance. As Tomas pointed, it can depend
on change of some settings like pool sizes etc. It can also depend on
count of data - for example you may see performance degradation just
when you have 10.000 users and each having 50 roles assigned to him
(just an example) etc etc.
In shortcut, there are lots of factors involved, so hard to say... :(
Marek
Regards,
Tomas
----- Original Message -----
> Hello Mario,
>
> Thanks for letting us know. I will try and have a look into it this week and
> try to isolate the cause of that regression.
>
>
> Regards,
> Tomas
>
>
> ----- Original Message -----
>> Hi,
>>
>> we're running nightly stress tests against our keycloak dev environment,
>> to monitor maximum throughput rates and average response times of
>> selected endpoints.
>>
>> After upgrading from KC 3.4.3 to 4.8.3, we noticed considerable dents in
>> our curves.
>>
>> For example:
>> Userinfo dropped from ~12k max. Requests per sec to ~7k; response times
>> increased from ~30ms to 45ms
>> Code flow (3 correlated requests): 1.5k -> 1k max. Requests per sec;
>> 100ms -> 150ms response time
>> Password Credentials Grant: 800 -> 600 max. Requests per sec; 300ms ->
>> 500ms response time
>>
>> We have another system running KC 4.5.0, which does not seem to suffer
>> from that performance decrease, so it was probably introduced with >
>> 4.5.
>>
>> Are there any known developments that might be the reason for our
>> observations?
>>
>> Thanks,
>> Mario.
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
2132
days inactive
2143
days old
3 comments
3 participants
participants (3)
-
keycloak-user＠imber.wien -
Marek Posolda -
Tomas Kyjovsky