2:12 p.m.
I'd like to have a hangout next week on this. There's really 4 areas to
focus on:
1. Set up a good performance test and get a baseline on current code.
2. Database caching. There are multiple approaches.
* Our caches will be read-mostly except for user session management.
* Write our own REST cache. My idea was to implement a new model
provider that delegates to the real underlying model provider. It would
send RESTful invalidation messages secured by keycloak in a clustered
environment. Maybe use Infinispan or some other local-only cache as the
backbone. I'm really really wary of using Infinispan (or other cache's)
clustering options. I'm not sure how well these guys work in cloud
clustered environments. And I'm not sure how well they've thought of
security.
* Use Hibernate 2nd level cache. Really wary of this as I don't know
how well Infinispan et. al. work in a cloud environment. I'd really
like something HTTP based and authenticated by Keycloak. I'm not sure
we'll be able to get the level of control here either that we need.
* Another reason for a custom cache, is, IMO, almost everything can be
stored in memory. Even users.
3. Clustering.
* Keycloak is not completely stateless. Specifically access codes are
stored in memory. We should be able to move this state information into
a cookie.
* User Session management is a huge issue as it is the only thing that
is write-mostly.
4. HTTP Cache-Control
* All themed static content should support a way to set up cache-control
headers.
* Admin REST api could do cache-control too if we had a way to determine
a version number or timestamp on model data. Not that important though
as, IMO, admin rest api isn't going to have a lot of volume.
--
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com
2:57 a.m.
On 29.5.2014 21:12, Bill Burke wrote:
I'd like to have a hangout next week on this. There's really
4 areas to
focus on:
1. Set up a good performance test and get a baseline on current code.
I've
already did some performance tests longer time ago when comparing
model implementations. Those are at "testsuite/performance" . These are
just Java tests, which are directly using model API (no end-to-end HTTP
tests ATM).
It's based on JMeter and it allows to configure number of concurrent
workers and number of "iterations" for each worker. For example test for
creating users with 10 workers and each worker creating 100 users means
1000 created users in total. Also in property files you can configure
number of users to create, realms where to create users etc.
Just figured out that tests doesn't work atm, but will try to possibly
fix and send PR also with some instructions.
2. Database caching. There are multiple approaches.
* Our caches will be read-mostly except for user session management.
I think that
in very big systems there might be also big number of user
registrations . And each user can change his account, so it might be big
number of edit actions, which can user done by himself (for example
linking/unlinking social account).
* Write our own REST cache. My idea was to implement a new model
provider that delegates to the real underlying model provider. It would
send RESTful invalidation messages secured by keycloak in a clustered
environment. Maybe use Infinispan or some other local-only cache as the
backbone. I'm really really wary of using Infinispan (or other cache's)
clustering options. I'm not sure how well these guys work in cloud
clustered environments. And I'm not sure how well they've thought of
security.
* Use Hibernate 2nd level cache. Really wary of this as I don't know
how well Infinispan et. al. work in a cloud environment. I'd really
like something HTTP based and authenticated by Keycloak. I'm not sure
we'll be able to get the level of control here either that we need.
* Another reason for a custom cache, is, IMO, almost everything can be
stored in memory. Even users.
Reusing infinispan at last as local-only cache would
be nice as it
solves many other things, like for example eviction/expiration . It has
also support for transactions (but if we want to use it, we may need to
add JTA support...)
Creating our own model provider will add good control, but might be time
consuming tasks . Tricky part is invalidation IMO. For example when you
delete application, you should drop also all application roles, role
memberships, scopes and scope memberships and userSession links to this
application. In the end, it might drop almost whole cache:-)
But probably we need to focus especially on events, which will happen
often (like creating new userSession or registering new user) and ensure
that there is no invalidation of more things then needed in these
cases. But for example registration of new user should invalidate all
user queries as well (I mean realm.getUsers() , realm.searchForUser and
realm.searchForUserByuAttribute).
3. Clustering.
* Keycloak is not completely stateless. Specifically access codes are
stored in memory. We should be able to move this state information into
a cookie.
* User Session management is a huge issue as it is the only thing that
is write-mostly.
There is also some state on SocialRequestManager .
4. HTTP Cache-Control
* All themed static content should support a way to set up cache-control
headers.
* Admin REST api could do cache-control too if we had a way to determine
a version number or timestamp on model data. Not that important though
as, IMO, admin rest api isn't going to have a lot of volume.
I would add "pagination" support. I think we need pagination at least
for users, probably also for userSessions on clients (I mean
clientModel.getUserSessions() ). Maybe also for AuditProvider query results.
Marek
6:25 a.m.
On 5/30/2014 3:57 AM, Marek Posolda wrote:
On 29.5.2014 21:12, Bill Burke wrote:
> I'd like to have a hangout next week on this. There's really 4 areas to
> focus on:
>
> 1. Set up a good performance test and get a baseline on current code.
I've already did some performance tests longer time ago when comparing
model implementations. Those are at "testsuite/performance" . These are
just Java tests, which are directly using model API (no end-to-end HTTP
tests ATM).
It's based on JMeter and it allows to configure number of concurrent
workers and number of "iterations" for each worker. For example test for
creating users with 10 workers and each worker creating 100 users means
1000 created users in total. Also in property files you can configure
number of users to create, realms where to create users etc.
Just figured out that tests doesn't work atm, but will try to possibly
fix and send PR also with some instructions.
>
> 2. Database caching. There are multiple approaches.
> * Our caches will be read-mostly except for user session management.
I think that in very big systems there might be also big number of user
registrations . And each user can change his account, so it might be big
number of edit actions, which can user done by himself (for example
linking/unlinking social account).
> * Write our own REST cache. My idea was to implement a new model
> provider that delegates to the real underlying model provider. It would
> send RESTful invalidation messages secured by keycloak in a clustered
> environment. Maybe use Infinispan or some other local-only cache as the
> backbone. I'm really really wary of using Infinispan (or other cache's)
> clustering options. I'm not sure how well these guys work in cloud
> clustered environments. And I'm not sure how well they've thought of
> security.
> * Use Hibernate 2nd level cache. Really wary of this as I don't know
> how well Infinispan et. al. work in a cloud environment. I'd really
> like something HTTP based and authenticated by Keycloak. I'm not sure
> we'll be able to get the level of control here either that we need.
> * Another reason for a custom cache, is, IMO, almost everything can be
> stored in memory. Even users.
Reusing infinispan at last as local-only cache would be nice as it
solves many other things, like for example eviction/expiration . It has
also support for transactions (but if we want to use it, we may need to
add JTA support...)
Creating our own model provider will add good control, but might be time
consuming tasks . Tricky part is invalidation IMO. For example when you
delete application, you should drop also all application roles, role
memberships, scopes and scope memberships and userSession links to this
application. In the end, it might drop almost whole cache:-)
The thing is, we can be as fine-grain as we want. If there is one
update in the realm, we can drop the whole cache of the realm. Our
custom invalidation could be really really simple in that if there is an
update, invalidate.
But probably we need to focus especially on events, which will
happen
often (like creating new userSession or registering new user) and ensure
that there is no invalidation of more things then needed in these
cases. But for example registration of new user should invalidate all
user queries as well (I mean realm.getUsers() , realm.searchForUser and
realm.searchForUserByuAttribute).
>
> 3. Clustering.
> * Keycloak is not completely stateless. Specifically access codes are
> stored in memory. We should be able to move this state information into
> a cookie.
> * User Session management is a huge issue as it is the only thing that
> is write-mostly.
There is also some state on SocialRequestManager .
>
> 4. HTTP Cache-Control
> * All themed static content should support a way to set up cache-control
> headers.
> * Admin REST api could do cache-control too if we had a way to determine
> a version number or timestamp on model data. Not that important though
> as, IMO, admin rest api isn't going to have a lot of volume.
>
I would add "pagination" support. I think we need pagination at least
for users, probably also for userSessions on clients (I mean
clientModel.getUserSessions() ). Maybe also for AuditProvider query results.
Ah thanks. Pagination yes.
--
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com
5:06 a.m.
Sent invite for Monday in Zimbra (15pm GMT). If that's not suitable let me know.
A few more areas to look at:
* Profile to find crappy code
* Tune FreeMarker and themes
* Sharding / sticky sessions - this is probably for later, but I'd like to discuss it
in the hangout
A few comments on database caching:
* We may need to do this in stages
- 2nd level cache for beta-2?
- Something more advanced in after 1.0.final
* I'd don't like the idea of us re-inventing the wheel
- Caching has a lot of complex considerations (node discovery, fail-over, node
recovery, invalidation, lan/wan, jta, etc.)
* I really like the idea of having a cache model provider that delegates to another model
for persistence / this is definitively the way to go IMO
* User sessions needs to separate considerations, specifically I think we can come up with
something clever to minimise the writes required
----- Original Message -----
From: "Bill Burke" <bburke(a)redhat.com>
To: keycloak-dev(a)lists.jboss.org
Sent: Thursday, 29 May, 2014 8:12:03 PM
Subject: [keycloak-dev] Next step performance
I'd like to have a hangout next week on this. There's really 4 areas to
focus on:
1. Set up a good performance test and get a baseline on current code.
2. Database caching. There are multiple approaches.
* Our caches will be read-mostly except for user session management.
* Write our own REST cache. My idea was to implement a new model
provider that delegates to the real underlying model provider. It would
send RESTful invalidation messages secured by keycloak in a clustered
environment. Maybe use Infinispan or some other local-only cache as the
backbone. I'm really really wary of using Infinispan (or other cache's)
clustering options. I'm not sure how well these guys work in cloud
clustered environments. And I'm not sure how well they've thought of
security.
* Use Hibernate 2nd level cache. Really wary of this as I don't know
how well Infinispan et. al. work in a cloud environment. I'd really
like something HTTP based and authenticated by Keycloak. I'm not sure
we'll be able to get the level of control here either that we need.
* Another reason for a custom cache, is, IMO, almost everything can be
stored in memory. Even users.
3. Clustering.
* Keycloak is not completely stateless. Specifically access codes are
stored in memory. We should be able to move this state information into
a cookie.
* User Session management is a huge issue as it is the only thing that
is write-mostly.
4. HTTP Cache-Control
* All themed static content should support a way to set up cache-control
headers.
* Admin REST api could do cache-control too if we had a way to determine
a version number or timestamp on model data. Not that important though
as, IMO, admin rest api isn't going to have a lot of volume.
--
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
6:29 a.m.
On 5/30/2014 6:06 AM, Stian Thorgersen wrote:
Sent invite for Monday in Zimbra (15pm GMT). If that's not
suitable let me know.
A few more areas to look at:
* Profile to find crappy code
* Tune FreeMarker and themes
* Sharding / sticky sessions - this is probably for later, but I'd like to discuss it
in the hangout
A few comments on database caching:
* We may need to do this in stages
- 2nd level cache for beta-2?
- Something more advanced in after 1.0.final
* I'd don't like the idea of us re-inventing the wheel
- Caching has a lot of complex considerations (node discovery, fail-over, node
recovery, invalidation, lan/wan, jta, etc.)
I know and agree and have experience with this. But, these traditional
caches just don't work well in many environments.
* I really like the idea of having a cache model provider that
delegates to another model for persistence / this is definitively the way to go IMO
* User sessions needs to separate considerations, specifically I think we can come up
with something clever to minimise the writes required
Yes, User sessions are a special case.
--
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com
7:17 a.m.
FIY I've started looking at caching FreeMarker templates and cache-control for static
content
----- Original Message -----
From: "Bill Burke" <bburke(a)redhat.com>
To: "Stian Thorgersen" <stian(a)redhat.com>
Cc: keycloak-dev(a)lists.jboss.org
Sent: Friday, 30 May, 2014 12:29:23 PM
Subject: Re: [keycloak-dev] Next step performance
On 5/30/2014 6:06 AM, Stian Thorgersen wrote:
> Sent invite for Monday in Zimbra (15pm GMT). If that's not suitable let me
> know.
>
> A few more areas to look at:
>
> * Profile to find crappy code
> * Tune FreeMarker and themes
> * Sharding / sticky sessions - this is probably for later, but I'd like to
> discuss it in the hangout
>
> A few comments on database caching:
>
> * We may need to do this in stages
> - 2nd level cache for beta-2?
> - Something more advanced in after 1.0.final
> * I'd don't like the idea of us re-inventing the wheel
> - Caching has a lot of complex considerations (node discovery,
> fail-over, node recovery, invalidation, lan/wan, jta, etc.)
I know and agree and have experience with this. But, these traditional
caches just don't work well in many environments.
> * I really like the idea of having a cache model provider that delegates to
> another model for persistence / this is definitively the way to go IMO
> * User sessions needs to separate considerations, specifically I think we
> can come up with something clever to minimise the writes required
>
Yes, User sessions are a special case.
--
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com
11:02 a.m.
Cache-control made a huge difference - halved the load time of the login screen!
Caching FreeMarker templates made much less of a difference (~10%). I'm a bit
disappointed as I thought it would be a bigger win (it did load and parse 2-3 files
templates for each page).
----- Original Message -----
From: "Stian Thorgersen" <stian(a)redhat.com>
To: "Bill Burke" <bburke(a)redhat.com>
Cc: keycloak-dev(a)lists.jboss.org
Sent: Friday, 30 May, 2014 1:17:43 PM
Subject: Re: [keycloak-dev] Next step performance
FIY I've started looking at caching FreeMarker templates and cache-control
for static content
----- Original Message -----
> From: "Bill Burke" <bburke(a)redhat.com>
> To: "Stian Thorgersen" <stian(a)redhat.com>
> Cc: keycloak-dev(a)lists.jboss.org
> Sent: Friday, 30 May, 2014 12:29:23 PM
> Subject: Re: [keycloak-dev] Next step performance
>
>
>
> On 5/30/2014 6:06 AM, Stian Thorgersen wrote:
> > Sent invite for Monday in Zimbra (15pm GMT). If that's not suitable let
> > me
> > know.
> >
> > A few more areas to look at:
> >
> > * Profile to find crappy code
> > * Tune FreeMarker and themes
> > * Sharding / sticky sessions - this is probably for later, but I'd like
> > to
> > discuss it in the hangout
> >
> > A few comments on database caching:
> >
> > * We may need to do this in stages
> > - 2nd level cache for beta-2?
> > - Something more advanced in after 1.0.final
> > * I'd don't like the idea of us re-inventing the wheel
> > - Caching has a lot of complex considerations (node discovery,
> > fail-over, node recovery, invalidation, lan/wan, jta, etc.)
>
> I know and agree and have experience with this. But, these traditional
> caches just don't work well in many environments.
>
> > * I really like the idea of having a cache model provider that delegates
> > to
> > another model for persistence / this is definitively the way to go IMO
> > * User sessions needs to separate considerations, specifically I think we
> > can come up with something clever to minimise the writes required
> >
>
> Yes, User sessions are a special case.
>
> --
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
3858
days inactive
3859
days old
6 comments
3 participants
participants (3)
-
Bill Burke -
Marek Posolda -
Stian Thorgersen