[keycloak-user] How to enable Infinispan cache for realms, users and user sessions in Keycloak 1.6.1?

Stian Thorgersen sthorger at redhat.com
Mon Nov 30 09:33:58 EST 2015


If you can try building master and enable trace logging for
"org.keycloak.models.cache.infinispan.InfinispanCacheUserProviderFactory".
It will print log statements when entries are removed/invalidated in the
cache.

On 30 November 2015 at 15:19, Bill Burke <bburke at redhat.com> wrote:

> Can you describe your load tests please?  What kind of authentication
> are you doing?  The only thing that would cause invalidations is HOTP.
>
> On 11/30/2015 9:13 AM, Lohitha Chiranjeewa wrote:
> > Bill, the problem here is that our load tests cannot go beyond a paltry
> > 15-20 concurrent threads with the default set up (invalidation caches
> > with SYNC mode - two HA AWS M3-Medium servers). Hence we're trying to
> > find ways to improve on the performance and the avg response time under
> > a load.
> >
> > On Mon, Nov 30, 2015 at 7:16 PM, Bill Burke <bburke at redhat.com
> > <mailto:bburke at redhat.com>> wrote:
> >
> >     BTW, besides being more scalable, we invalidate rather than
> replicate to
> >     avoid transmitting sensitive security data over the network.
> >
> >     On 11/30/2015 8:29 AM, Lohitha Chiranjeewa wrote:
> >     > Glad if you could look into the above and check if there are app
> level
> >     > issues related to invalidations, thanks.
> >     >
> >     > One way to get around this through config changes is to make the
> >     > relevant cache modes 'ASYNC' and set up suitable values for
> 'queue-size'
> >     > and 'queue-flush-interval' depending on your needs. Then the
> >     > invalidations won't happen with each and every call.
> >     >
> >     > Obviously there would be problems if caches aren't invalidated in
> time
> >     > with the above set up, it's up to the devs to come up with a
> suitable
> >     > set of configs to cater to their needs.
> >     >
> >     >
> >     > Regards,
> >     > Lohitha.
> >     >
> >     >
> >     >
> >     > On Mon, Nov 30, 2015 at 1:08 PM, Stian Thorgersen <
> sthorger at redhat.com <mailto:sthorger at redhat.com>
> >     > <mailto:sthorger at redhat.com <mailto:sthorger at redhat.com>>> wrote:
> >     >
> >     >     It is not expected behavior. Users should only be invalidated
> if
> >     >     changes are made.
> >     >
> >     >     On 30 November 2015 at 07:01, Lohitha Chiranjeewa <
> kalc04 at gmail.com <mailto:kalc04 at gmail.com>
> >     >     <mailto:kalc04 at gmail.com <mailto:kalc04 at gmail.com>>> wrote:
> >     >
> >     >         Just tested the same flow extensively with Keycloak 1.2.0
> as
> >     >         well, and it seems the behavior is the same. Lots of MySQL
> >     >         select queries getting executed with each call. Seems
> there's a
> >     >         number of unnecessary cache invalidations going on which
> causes
> >     >         Keycloak to fetch data from the DB over and over.
> >     >
> >     >         Could you please confirm if this is the expected behavior?
> As
> >     >         far as I can see there's considerable performance
> degradation
> >     >         due to unnecessary cache invalidations.
> >     >
> >     >
> >     >         On Fri, Nov 27, 2015 at 8:42 PM, Lohitha Chiranjeewa
> >     >         <kalc04 at gmail.com <mailto:kalc04 at gmail.com> <mailto:
> kalc04 at gmail.com
> >     <mailto:kalc04 at gmail.com>>> wrote:
> >     >
> >     >             I'm invoking the following API call:
> >     >             https://
> {host}/auth/admin/realms/xxxxx/users/55ffe851-2d94-460e-88b9-bc7340531b56
> >     >             (same realm and user ID over and over again). The two
> HA
> >     >             server(s) are idle apart from serving those calls. And
> I'm
> >     >             seeing the following SQL getting logged in my MySQL
> log for
> >     >             each and every call:
> >     >
> >     >             select userentity0_.ID as ID1_42_,
> >     >             userentity0_.CREATED_TIMESTAMP as CREATED_2_42_,
> >     >             userentity0_.EMAIL as EMAIL3_42_,
> >     >             userentity0_.EMAIL_CONSTRAINT as EMAIL_CO4_42_,
> >     >             userentity0_.EMAIL_VERIFIED as EMAIL_VE5_42_,
> >     >             userentity0_.ENABLED as ENABLED6_42_,
> >     >             userentity0_.federation_link as federati7_42_,
> >     >             userentity0_.FIRST_NAME as FIRST_NA8_42_,
> >     >             userentity0_.LAST_NAME as LAST_NAM9_42_,
> >     >             userentity0_.REALM_ID as REALM_I10_42_,
> >     >             userentity0_.SERVICE_ACCOUNT_CLIENT_LINK as
> SERVICE11_42_,
> >     >             userentity0_.TOTP as TOTP12_42_, userentity0_.USERNAME
> as
> >     >             USERNAM13_42_ from USER_ENTITY userentity0_ where
> >     >             userentity0_.ID='55ffe851-2d94-460e-88b9-bc7340531b56'
> and
> >     >             userentity0_.REALM_ID='xxxxx'
> >     >
> >     >
> >     >
> >     >             On Fri, Nov 27, 2015 at 8:03 PM, Stian Thorgersen
> >     >             <sthorger at redhat.com <mailto:sthorger at redhat.com>
> >     <mailto:sthorger at redhat.com <mailto:sthorger at redhat.com>>> wrote:
> >     >
> >     >                 Strange - what are you doing and what are the SQL
> queries?
> >     >
> >     >                 On 27 November 2015 at 15:23, Lohitha Chiranjeewa
> >     >                 <kalc04 at gmail.com <mailto:kalc04 at gmail.com>
> <mailto:kalc04 at gmail.com
> >     <mailto:kalc04 at gmail.com>>> wrote:
> >     >
> >     >                     Yes Stian, that I understand. But the problem
> here
> >     >                     is even if I execute continuous user retrieval
> calls
> >     >                     (same user - no other functionality in
> between),
> >     >                     still MySQL select queries get executed for
> each
> >     >                     call. So there lies an issue isn't it?
> >     >
> >     >
> >     >                     On Fri, Nov 27, 2015 at 6:48 PM, Stian
> Thorgersen
> >      >                     <sthorger at redhat.com
> >     <mailto:sthorger at redhat.com> <mailto:sthorger at redhat.com
> >     <mailto:sthorger at redhat.com>>>
> >     >                     wrote:
> >     >
> >     >                         Things are still fetched from MySQL.
> Realms,
> >     >                         clients, users, etc.. are then kept in the
> >     >                         cache, but if it changes it's re-loaded
> from
> >     >                         MySQL. We use an invalidation cache, not a
> >     >                         distributed cache.
> >     >
> >     >                         On 27 November 2015 at 14:04, Lohitha
> >     >                         Chiranjeewa <kalc04 at gmail.com <mailto:
> kalc04 at gmail.com>
> >     >                         <mailto:kalc04 at gmail.com <mailto:
> kalc04 at gmail.com>>> wrote:
> >     >
> >     >                             What I mean is, if it were working, I
> >     >                             shouldn't see mysql queries getting
> executed
> >     >                             right? So my guess is data is still
> fetched
> >     >                             from the db instead of the cache.
> >     >
> >     >                             On Nov 27, 2015 5:52 PM, "Stian
> Thorgersen"
> >     >                             <sthorger at redhat.com <mailto:
> sthorger at redhat.com>
> >     >                             <mailto:sthorger at redhat.com <mailto:
> sthorger at redhat.com>>> wrote:
> >     >
> >     >                                 Yup, so it's working now?
> >     >
> >     >                                 On 27 November 2015 at 13:20,
> Lohitha
> >     >                                 Chiranjeewa <kalc04 at gmail.com
> <mailto:kalc04 at gmail.com>
> >     >                                 <mailto:kalc04 at gmail.com <mailto:
> kalc04 at gmail.com>>> wrote:
> >     >
> >     >                                     Apologies, keycloak-server.json
> >     >                                     entries should change to:
> >     >
> >     >                                          "realm": {
> >     >                                              "provider": "jpa"
> >     >                                          },
> >     >
> >     >                                          "user": {
> >     >                                              "provider": "jpa"
> >     >                                          },
> >     >
> >     >                                          "userSessionPersister": {
> >     >                                              "provider": "jpa"
> >     >                                          },
> >     >
> >     >                                     On Fri, Nov 27, 2015 at 5:49
> PM,
> >     >                                     Lohitha Chiranjeewa
> >     >                                     <kalc04 at gmail.com <mailto:
> kalc04 at gmail.com>
> >      >                                     <mailto:kalc04 at gmail.com
> >     <mailto:kalc04 at gmail.com>>> wrote:
> >      >
> >      >                                         Hi Stian,
> >      >
> >      >                                         As per the migration
> guide, I
> >      >                                         should have Infinispan up
> and
> >      >                                         running for realms, users
> and
> >      >                                         user sessions without
> >     doing any
> >      >                                         specific changes.
> >      >                                         keycloak-server.json was
> >      >                                         reverted back to have the
> >      >                                         following entries:
> >      >                                         ...
> >      >                                              "realm": {
> >      >                                                  "provider":
> >     "infinispan"
> >      >                                              },
> >      >
> >      >                                              "user": {
> >      >                                                  "provider":
> >     "infinispan"
> >      >                                              },
> >      >
> >      >
> >     "userSessionPersister": {
> >      >                                                  "provider":
> >     "infinispan"
> >      >                                              },
> >      >                                         ...
> >      >
> >      >                                         In the Admin Console I
> >     have both
> >      >                                         Realm Cache and User Cache
> >      >                                         enables. I see certain
> >      >                                         Infinispan related logs
> >     getting
> >      >                                         logged as well.
> >      >
> >      >                                         However, at the same
> >     time, I see
> >      >                                         MySQL queries getting
> >     executed
> >      >                                         for all user retrieval API
> >      >                                         invocations (even if the
> same
> >      >                                         user is retrieved
> >     continuously):
> >      >                                         ...
> >      >                                         select userentity0_.ID as
> >      >                                         ID1_42_,
> >      >
> >       userentity0_.CREATED_TIMESTAMP
> >      >                                         as CREATED_2_42_,
> >      >                                         userentity0_.EMAIL as
> >      >                                         EMAIL3_42_,
> >      >
> >       userentity0_.EMAIL_CONSTRAINT as
> >      >                                         EMAIL_CO4_42_,
> >      >
> >       userentity0_.EMAIL_VERIFIED as
> >      >                                         EMAIL_VE5_42_,
> >      >                                         userentity0_.ENABLED as
> >      >                                         ENABLED6_42_,
> >      >
> >       userentity0_.federation_link as
> >      >                                         federati7_42_,
> >      >                                         userentity0_.FIRST_NAME as
> >      >                                         FIRST_NA8_42_,
> >      >                                         userentity0_.LAST_NAME as
> >      >                                         LAST_NAM9_42_,
> >      >                                         userentity0_.REALM_ID as
> >      >                                         REALM_I10_42_,
> >      >
> >       userentity0_.SERVICE_ACCOUNT_CLIENT_LINK
> >      >                                         as SERVICE11_42_,
> >      >                                         userentity0_.TOTP as
> >     TOTP12_42_,
> >      >                                         userentity0_.USERNAME as
> >      >                                         USERNAM13_42_ from
> >     USER_ENTITY
> >      >                                         userentity0_ where
> >      >
> >       userentity0_.ID='55ffe851-2d94-460e-88b9-bc7340531b56'
> >      >                                         and
> >     userentity0_.REALM_ID='xxxxx'
> >      >                                         ...
> >      >
> >      >                                         So it seems something is
> >     wrong
> >      >                                         here. Could you point out
> any
> >      >                                         areas that I could
> >     further look
> >      >                                         into?
> >      >
> >      >
> >      >                                         Regards,
> >      >                                         Lohitha.
> >      >
> >      >                                         On Thu, Nov 26, 2015 at
> >     7:58 PM,
> >      >                                         Stian Thorgersen
> >      >                                         <sthorger at redhat.com
> >     <mailto:sthorger at redhat.com>
> >     >                                         <mailto:
> sthorger at redhat.com <mailto:sthorger at redhat.com>>> wrote:
> >     >
> >     >                                             Please read the
> migration guide
> >     >
> >     >                                             On 26 November 2015 at
> >     >                                             14:53, Lohitha
> Chiranjeewa
> >     >                                             <kalc04 at gmail.com
> <mailto:kalc04 at gmail.com>
> >      >
> >       <mailto:kalc04 at gmail.com <mailto:kalc04 at gmail.com>>>
> >      >                                             wrote:
> >      >
> >      >                                                 Hi,
> >      >
> >      >                                                 We're in the
> >     process of
> >      >                                                 assessing the
> >     impact on
> >      >                                                 upgrading from
> >     Keycloak
> >      >                                                 1.2.0 to 1.6.1.
> >     We came
> >      >                                                 across an issue
> when
> >      >                                                 trying to enable
> >      >                                                 Infinispan cache
> >     through
> >      >                                                 the
> >     keycloak-server.json
> >      >                                                 file as we used
> >     to do in
> >      >                                                 1.2.0.
> >      >
> >      >                                                 We have the
> following
> >      >                                                 entries in 1.6.1:
> >      >                                                      "realm": {
> >      >
> "provider":
> >      >                                                 "infinispan"
> >      >                                                      },
> >      >
> >      >                                                      "user": {
> >      >
> "provider":
> >      >                                                 "infinispan"
> >      >                                                      },
> >      >
> >      >
> >      >
> >       "userSessionPersister": {
> >      >
> "provider":
> >      >                                                 "infinispan"
> >      >                                                      },
> >      >                                                 .........
> >      >
> >      >
> >       "connectionsInfinispan": {
> >      >
> >     "default" : {
> >      >
> >      >                                                 "cacheContainer" :
> >      >
> >       "java:comp/env/infinispan/Keycloak"
> >      >                                                          }
> >      >                                                      }
> >      >
> >      >                                                 All
> configurations in
> >      >                                                 1.6.1
> >     standalone-ha.xml
> >      >                                                 file remains
> >     comparable
> >      >                                                 (and correct to
> >     the best
> >      >                                                 of our knowledge)
> >     with
> >      >                                                 the ones in 1.2.0.
> >      >
> >      >                                                 With the above
> >     configs,
> >      >                                                 when we start the
> >      >                                                 Keycloak service
> the
> >      >                                                 following
> >     error(s) get
> >      >                                                 logged:
> >      >
> >      >                                                 18:03:31,610 ERROR
> >      >
> >       [org.jboss.msc.service.fail]
> >      >                                                 (ServerService
> Thread
> >      >                                                 Pool -- 64)
> >     MSC000001:
> >      >                                                 Failed to start
> >     service
> >      >
> >       jboss.undertow.deployment.default-server.default-host./auth:
> >      >
> >       org.jboss.msc.service.StartException
> >      >                                                 in service
> >      >
> >       jboss.undertow.deployment.default-server.default-host./auth:
> >      >
> >       java.lang.RuntimeException:
> >      >                                                 Failed to
> construct
> >      >                                                 public
> >      >
> >
>  org.keycloak.services.resources.KeycloakApplication(javax.servlet.ServletContext,org.jboss.resteasy.core.Dispatcher)
> >      >                                                      at
> >      >
> >
>  org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
> >      >                                                      at
> >      >
> >
>  java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >      >                                                 [rt.jar:1.7.0_45]
> >      >                                                      at
> >      >
> >       java.util.concurrent.FutureTask.run(FutureTask.java:262)
> >      >                                                 [rt.jar:1.7.0_45]
> >      >                                                      at
> >      >
> >
>  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> >      >                                                 [rt.jar:1.7.0_45]
> >      >                                                      at
> >      >
> >
>  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> >      >                                                 [rt.jar:1.7.0_45]
> >      >                                                      at
> >      >
> >       java.lang.Thread.run(Thread.java:744)
> >      >                                                 [rt.jar:1.7.0_45]
> >      >                                                      at
> >      >
> >       org.jboss.threads.JBossThread.run(JBossThread.java:320)
> >      >
> >       [jboss-threads-2.2.0.Final.jar:2.2.0.Final]
> >      >                                                 Caused by:
> >      >
> >       java.lang.RuntimeException:
> >      >                                                 Failed to
> construct
> >      >                                                 public
> >      >
> >
>  org.keycloak.services.resources.KeycloakApplication(javax.servlet.ServletContext,org.jboss.resteasy.core.Dispatcher)
> >      >                                                      at
> >      >
> >
>  org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:160)
> >      >                                                      at
> >      >
> >
>  org.jboss.resteasy.spi.ResteasyProviderFactory.createProviderInstance(ResteasyProviderFactory.java:2211)
> >      >                                                      at
> >      >
> >
>  org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:295)
> >      >                                                      at
> >      >
> >
>  org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:236)
> >      >                                                      at
> >      >
> >
>  org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:112)
> >      >                                                      at
> >      >
> >
>  org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36)
> >      >                                                      at
> >      >
> >
>  io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:117)
> >      >                                                      at
> >      >
> >
>  org.wildfly.extension.undertow.security.RunAsLifecycleInterceptor.init(RunAsLifecycleInterceptor.java:78)
> >      >                                                      at
> >      >
> >
>  io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:103)
> >      >                                                      at
> >      >
> >
>  io.undertow.servlet.core.ManagedServlet$DefaultInstanceStrategy.start(ManagedServlet.java:230)
> >      >                                                      at
> >      >
> >
>  io.undertow.servlet.core.ManagedServlet.createServlet(ManagedServlet.java:131)
> >      >                                                      at
> >      >
> >
>  io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:511)
> >      >                                                      at
> >      >
> >
>  org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:101)
> >      >                                                      at
> >      >
> >
>  org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
> >      >                                                      ... 6 more
> >      >                                                 Caused by:
> >      >
> >       java.lang.RuntimeException:
> >      >                                                 Failed to find
> >     provider
> >      >                                                 infinispan for
> realm
> >      >                                                      at
> >      >
> >
>  org.keycloak.services.DefaultKeycloakSessionFactory.init(DefaultKeycloakSessionFactory.java:66)
> >      >                                                      at
> >      >
> >
>  org.keycloak.services.resources.KeycloakApplication.createSessionFactory(KeycloakApplication.java:162)
> >      >                                                      at
> >      >
> >
>  org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:62)
> >      >                                                      at
> >      >
> >       sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >      >                                                 Method)
> >     [rt.jar:1.7.0_45]
> >      >                                                      at
> >      >
> >
>  sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> >      >                                                 [rt.jar:1.7.0_45]
> >      >                                                      at
> >      >
> >
>  sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> >      >                                                 [rt.jar:1.7.0_45]
> >      >                                                      at
> >      >
> >       java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> >      >                                                 [rt.jar:1.7.0_45]
> >      >                                                      at
> >      >
> >
>  org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:148)
> >      >                                                      ... 19 more
> >      >
> >      >
> >      >                                                 Is the new way to
> >     enable
> >      >                                                 Infinispan
> >     different to
> >      >                                                 what we had
> >     earlier? If
> >      >                                                 so, can someone
> >     please
> >      >                                                 point out the
> >     correct way?
> >      >
> >      >
> >      >                                                 Regards,
> >      >                                                 Lohitha.
> >      >
> >      >
> >      >
> >      >
> >       _______________________________________________
> >      >                                                 keycloak-user
> >     mailing list
> >      > keycloak-user at lists.jboss.org <mailto:
> keycloak-user at lists.jboss.org>
> >      >
> >       <mailto:keycloak-user at lists.jboss.org
> >     <mailto:keycloak-user at lists.jboss.org>>
> >      > https://lists.jboss.org/mailman/listinfo/keycloak-user
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     > _______________________________________________
> >     > keycloak-user mailing list
> >     >keycloak-user at lists.jboss.org <mailto:keycloak-user at lists.jboss.org
> >
> >     >https://lists.jboss.org/mailman/listinfo/keycloak-user
> >     >
> >
> >     --
> >     Bill Burke
> >     JBoss, a division of Red Hat
> >     http://bill.burkecentral.com
> >     _______________________________________________
> >     keycloak-user mailing list
> >     keycloak-user at lists.jboss.org <mailto:keycloak-user at lists.jboss.org>
> >     https://lists.jboss.org/mailman/listinfo/keycloak-user
> >
> >
>
> --
> Bill Burke
> JBoss, a division of Red Hat
> http://bill.burkecentral.com
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-user/attachments/20151130/ad883c5f/attachment-0001.html 


More information about the keycloak-user mailing list