[keycloak-dev] Next step performance

[Bill Burke]

On 5/30/2014 3:57 AM, Marek Posolda wrote:
> On 29.5.2014 21:12, Bill Burke wrote:
>> I'd like to have a hangout next week on this.  There's really 4 areas to
>> focus on:
>>
>> 1. Set up a good performance test and get a baseline on current code.
> I've already did some performance tests longer time ago when comparing
> model implementations. Those are at "testsuite/performance" . These are
> just Java tests, which are directly using model API (no end-to-end HTTP
> tests ATM).
>
> It's based on JMeter and it allows to configure number of concurrent
> workers and number of "iterations" for each worker. For example test for
> creating users with 10 workers and each worker creating 100 users means
> 1000 created users in total. Also in property files you can configure
> number of users to create, realms where to create users etc.
>
> Just figured out that tests doesn't work atm, but will try to possibly
> fix and send PR also with some instructions.
>
>>
>> 2. Database caching.  There are multiple approaches.
>> * Our caches will be read-mostly except for user session management.
> I think that in very big systems there might be also big number of user
> registrations . And each user can change his account, so it might be big
> number of edit actions, which can user done by himself (for example
> linking/unlinking social account).
>> * Write our own REST cache.  My idea was to implement a new model
>> provider that delegates to the real underlying model provider.  It would
>> send RESTful invalidation messages secured by keycloak in a clustered
>> environment.  Maybe use Infinispan or some other local-only cache as the
>> backbone.  I'm really really wary of using Infinispan (or other cache's)
>> clustering options.  I'm not sure how well these guys work in cloud
>> clustered environments. And I'm not sure how well they've thought of
>> security.
>> * Use Hibernate 2nd level cache.  Really wary of this as I don't know
>> how well Infinispan et. al. work in a cloud environment.  I'd really
>> like something HTTP based and authenticated by Keycloak.  I'm not sure
>> we'll be able to get the level of control here either that we need.
>> * Another reason for a custom cache, is, IMO, almost everything can be
>> stored in memory. Even users.
> Reusing infinispan at last as local-only cache would be nice as it
> solves many other things, like for example eviction/expiration . It has
> also support for transactions (but if we want to use it, we may need to
> add JTA support...)
>
> Creating our own model provider will add good control, but might be time
> consuming tasks . Tricky part is invalidation IMO. For example when you
> delete application, you should drop also all application roles, role
> memberships, scopes and scope memberships and userSession links to this
> application. In the end, it might drop almost whole cache:-)

The thing is, we can be as fine-grain as we want.  If there is one 
update in the realm, we can drop the whole cache of the realm.  Our 
custom invalidation could be really really simple in that if there is an 
update, invalidate.


> But probably we need to focus especially on events, which will happen
> often (like creating new userSession or registering new user) and ensure
> that there is no invalidation of more things then needed in these
> cases.  But for example registration of new user should invalidate all
> user queries as well (I mean realm.getUsers() , realm.searchForUser and
> realm.searchForUserByuAttribute).
>
>>
>> 3. Clustering.
>> * Keycloak is not completely stateless.  Specifically access codes are
>> stored in memory.  We should be able to move this state information into
>> a cookie.
>> * User Session management is a huge issue as it is the only thing that
>> is write-mostly.
> There is also some state on SocialRequestManager .
>>
>> 4. HTTP Cache-Control
>> * All themed static content should support a way to set up cache-control
>> headers.
>> * Admin REST api could do cache-control too if we had a way to determine
>> a version number or timestamp on model data.  Not that important though
>> as, IMO, admin rest api isn't going to have a lot of volume.
>>
> I would add "pagination" support. I think we need pagination at least
> for users, probably also for userSessions on clients (I mean
> clientModel.getUserSessions() ). Maybe also for AuditProvider query results.
>

Ah thanks.  Pagination yes.


-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com