[keycloak-dev] client query caches getting complicated

[Bill Burke]

On 2/17/2016 3:09 PM, Marek Posolda wrote:
> On 17/02/16 18:11, Bill Burke wrote:
>> Currently, adding or deleting a client, or updating a realm causes
>> invalidation/eviction of the realm and all clients in that realm. To
>> make matters worse, the next time the realm is accessed, it queries and
>> loads each client and its relationships.  Why do we do this?  When a
>> realm invalidation happens, the cache has no idea if the realm is just
>> being updated or removed entirely from the DB. With a realm removal, you
>> also need to evict the cache of all clients within the realm.  So, a
>> cached realm MUST have a list of all clients within it.  As a result,
>> the more clients that get added, keycloak gets slower and slower.
>> Eventually though the cache stabilizes after inserts/update/deletes
>> subside and we get back to normal performance, but you can see a nasty
>> blip for a little bit.
> We don't need to eagerly preload all clients when realm is loaded. 
> Infinispan has streaming/predicate API and we are using it in many 
> places (see InfinispanUserSessionProvider and all the stuff in package 
> org.keycloak.models.sessions.infinispan.stream ). So when entry is 
> invalidated, we can have cacheListener, which will query infinispan to 
> return all cached clients of the realm and remove them. I can see that 
> we already have listener. So possibly we can just change this line to 
> use predicate query: 
> https://github.com/keycloak/keycloak/blob/master/model/infinispan/src/main/java/org/keycloak/models/cache/infinispan/locking/LockingCacheRealmProviderFactory.java#L141
>
>

Lol, i was sort of kind of doing that doing cache.values().  This makes 
it much easier with much less refactoring.

> Other option is, that we can try to optimize eager preload and see if 
> we can reduce number of SQL queries at startup. For example, it seems 
> that CachedClient constructor sends SQL query to preload just realm 
> roles. And then each client sends another SQL query to preload his 
> client roles (method CachedClient.cacheRoles ). Couldn't we instead 
> send one SQL query to load all roles of realm (both realm and client 
> roles)? Maybe we can even send one big SQL query to load realm with 
> all it's objects :-) Possibly some FetchType.EAGER in Hibernate 
> entities could help here.
>

yeah, we should start looking into stuff like this.    We can at least 
reduce the number of sql calls for caching a client or realm 
individually.  I'm not sure how feasible it is though or how much better 
it is.  Realms and clients contain more than a few one to many 
relationships which would make the query result HUGE if I remember how 
joins work.

-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20160217/f3dc4d00/attachment.html