[keycloak-user] token introspection

Thu Aug 10 09:45:17 EDT 2017

You don't necessarily 'need' it, but it makes the project an order of
magnitude more complicated to understand.  The adapters are magic, but if
we aren't already familiar with something like Spring, then we really have
no way to fully understand what is going on. The other adapters such as
undertow as equally as intense. We don't need to understand all
infrastructure of course, but given there is an integration into an
existing workflow it makes it hard to use.

I would contribute a simple / clean Java example but I am still not sure
what that would even mean. Most threads I see around inspection say just
use XXX and don't worry. But I have found that to be dubious.

The aerogear team used using a Auth0 JWT inspection library which seems
like it is working.

The project is very well documented and there are tons of examples, but I
see every 2 weeks or so someone asks about token introspection as they are
trying to learn the best way to use Keycloak and basically get 3 responses
in until they get a 'dont worry about it' response, which really doesn't
help as it is rare to solve a more complicated problem without
understanding the basic problem.

We get the fear of missing using token inspection, but it sucks as a user
not to be able to get a clear example of 'Here is how you used to do it
with manually token inspection, now compare to the adapters' as opposed to
'read this intense doc and just simply integrate this adapter which you
don't understand' which has been what I have personally seen in digging
through 3 years of emails. Or the even better - "just google for industry
best practices on token inspection and use the existing libraries". For you
on the team it probably seems like such a trivial use case not even worth
mentioning but if you are new to Oauth/Keycloak/JWT and adapters, a simple
use case even if it was absolutely evil and going to cause WW3 would be
useful. Even as a blog post.

The use case is, I use keycloak to do a social sign-in, I get the token
back, using only self-contained Java (so calling libraries but not using
any json configs or adapters), how would I inspect the token to pull out
the email address and verify that token is still valid and actually came
from my keycloak server instead of being spoofed?

I know this is what adapters/json configs do for you (at least in theory, I
never got adapters to work), and there are existing libraries, but an all
in one laid out example direct from keycloak team would be awesome!

On Thu, Aug 10, 2017 at 8:58 AM, <keycloak-user-request at lists.jboss.org>
wrote:

> Send keycloak-user mailing list submissions to
>         keycloak-user at lists.jboss.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://lists.jboss.org/mailman/listinfo/keycloak-user
> or, via email, send a message with subject or body 'help' to
>         keycloak-user-request at lists.jboss.org
>
> You can reach the person managing the list at
>         keycloak-user-owner at lists.jboss.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of keycloak-user digest..."
>
>
> Today's Topics:
>
>    1. Re: DB deadlock for concurrent logins (Vikrant Singh)
>    2. Re: token introspection (Pedro Igor Silva)
>    3. Re: JGroups failure: failed submitting DONT_BUNDLE message to
>       thread pool (Hynek Mlnarik)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 10 Aug 2017 17:06:33 +0530
> From: Vikrant Singh <vikrant02.work at gmail.com>
> Subject: Re: [keycloak-user] DB deadlock for concurrent logins
> To: "keycloak-us." <keycloak-user at lists.jboss.org>
> Message-ID:
>         <CACackctxEZcXL37cAMuS7ajhmdHyr_SHNGBT=qaz6Nw0W428oQ at mail.
> gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> Yes, but password migration should only be happening during first login. I
> have checked the CREDENTIAL table in DB and user have already migrated to
> new hashing algorithm and no of iterations, but still keycloak is trying to
> run Update query for each login attempt.
>
> -Vikrant
>
> On Thu, Aug 10, 2017 at 4:18 PM, <keycloak-user-request at lists.jboss.org>
> wrote:
> >
> >
> > I believe that the default hashing algorithm has changed to SHA-256 as
> > SHA-1 was recently in the news for being able to create hash
> > collisions.  Because of this change, each login will update the password
> > hash stored with the 1st login of the user.
> >
> > Not sure why MariaDB would deadlock.  Don't know enough about how that
> > database performs locks.  It is trying to update a foreign key that has
> > an index associated with it.  Maybe that has something to do with it.
> >
> >
> >
> > On 8/9/17 5:35 PM, Vikrant Singh wrote:
> > > Hi,
> > >
> > > I am Running Keycloak 3.2.1.Final on openshift platform with MariaDB
> > 10.2.7
> > > for DB, recently upgraded from 3.1.0.Final.
> > >
> > > Deployment is consist of 3 keycloak servers along with 3 DB instances.
> As
> > > part of kubernetes rediness check, a token is requested for a local
> user
> > in
> > > master realm every 10 sec. The concurrent token request for same user
> is
> > > causing the deadlock exception in DB. Following is the exception being
> > > logged in keycloak.
> > >
> > >
> > > Caused by: java.sql.SQLException: Deadlock found when trying to get
> > > lock; try restarting transaction
> > >
> > > Query is: select userentity0_.ID as ID1_71_,
> > > userentity0_.CREATED_TIMESTAMP as CREATED_2_71_, userentity0_.EMAIL as
> > > EMAIL3_71_, userentity0_.EMAIL_CONSTRAINT as EMAIL_CO4_71_,
> > > userentity0_.EMAIL_VERIFIED as EMAIL_VE5_71_, userentity0_.ENABLED as
> > > ENABLED6_71_, userentity0_.FEDERATION_LINK as FEDERATI7_71_,
> > > userentity0_.FIRST_NAME as FIRST_NA8_71_, userentity0_.LAST_NAME as
> > > LAST_NAM9_71_, userentity0_.REALM_ID as REALM_I10_71_,
> > > userentity0_.SERVICE_ACCOUNT_CLIENT_LINK as SERVICE11_71_,
> > > userentity0_.USERNAME as USERNAM12_71_ from USER_ENTITY userentity0_
> > > where userentity0_.ID=? and userentity0_.REALM_ID=?, parameters
> > > ['ddafa525-baae-4c40-98f8-08c25a23f2c6','master']
> > >
> > >       at org.mariadb.jdbc.internal.util.LogQueryTool.
> exceptionWithQuery(
> > LogQueryTool.java:146)
> > >
> > >       at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.
> > executeQuery(AbstractQueryProtocol.java:221)
> > >
> > >       at org.mariadb.jdbc.MariaDbPreparedStatementClient
> > .executeInternal(MariaDbPreparedStatementClient.java:218)
> > >
> > >       ... 76 more
> > >
> > >
> > > Caused by: java.sql.SQLException: Lock wait timeout exceeded; try
> > > restarting transaction
> > >
> > > Query is: update CREDENTIAL set ALGORITHM=?, COUNTER=?,
> > > CREATED_DATE=?, DEVICE=?, DIGITS=?, HASH_ITERATIONS=?, PERIOD=?,
> > > SALT=?, TYPE=?, USER_ID=?, VALUE=? where ID=?, parameters
> > > ['pbkdf2-sha256',0,1501750736628,<null>,0,27500,0,<bytearray:???7'3^
> > > .??LT???>,'password','ddafa525-baae-4c40-98f8-08c25a23f2c6','
> > Hdpx8Zg5Ec8M9qVUp+Ylwlje+nhcGAzVPStF6/cvrqZghTeby048b8d3uqExfzS0of/
> > 9Quwx9CROGKTC685Tpw==','5929a82b-542c-4597-b3eb-524d74e58919']
> > >
> > >       at org.mariadb.jdbc.internal.util.LogQueryTool.
> exceptionWithQuery(
> > LogQueryTool.java:146)
> > >
> > >       at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.
> > executeQuery(AbstractQueryProtocol.java:221)
> > >
> > >       at org.mariadb.jdbc.MariaDbPreparedStatementClient
> > .executeInternal(MariaDbPreparedStatementClient.java:218)
> > >
> > >       ... 78 more
> > >
> > >
> > > Why keycloak is trying to update the user credential for every login.
> > > and why is deadlock occurring? Any help truly appreciated.
> > >
> > >
> > > Thanks,
> > >
> > > Vikrant
> > > _______________________________________________
> > > keycloak-user mailing list
> > > keycloak-user at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/keycloak-user
> >
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 10 Aug 2017 08:53:23 -0300
> From: Pedro Igor Silva <psilva at redhat.com>
> Subject: Re: [keycloak-user] token introspection
> To: Simon Payne <simonpayne58 at gmail.com>
> Cc: keycloak-user <keycloak-user at lists.jboss.org>
> Message-ID:
>         <CAJrcDBfFW3M4yauqgn=eG8Wnp=MnPkpQ4OFjeyoUZPLowm=1EQ at mail.
> gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> No, we don't. Like Bill said, you don't really need it. Basically, what we
> support is described in docs [1].
>
> [1]
> http://www.keycloak.org/docs/3.1/authorization_services/
> topics/enforcer/keycloak-enforcement-filter.html
>
>
> On Thu, Aug 10, 2017 at 6:11 AM, Simon Payne <simonpayne58 at gmail.com>
> wrote:
>
> > do we have token introspection implemented in any of the client adapters
> > (other than spring boot)?
> >
> > thanks
> >
> >
> > On Wed, Aug 9, 2017 at 9:50 AM, Simon Payne <simonpayne58 at gmail.com>
> > wrote:
> >
> > > thanks Pedro,
> > >
> > > however, i think our use cases are not exactly the same.  it appears
> your
> > > component is set to allow authentication of user where mine is bearer
> > only.
> > >
> > > the only other differences i can see between our projects is that i am
> > > running gradle with keycloak 3.2.0 and that i have also added compile(
> > > 'org.keycloak:keycloak-authz-client:3.2.0.CR1')
> > >
> > > Lucian, i don't have a project which i can share at the moment as other
> > > code is included, if you would still like to see something i can make a
> > > shareable version.
> > >
> > > Thanks
> > >
> > >
> > > On Tue, Aug 8, 2017 at 8:57 PM, Pedro Igor Silva <psilva at redhat.com>
> > > wrote:
> > >
> > >> Hey Lucian, we have this https://github.com/keycloak/ke
> > >> ycloak-quickstarts/tree/latest/app-authz-springboot.
> > >>
> > >> On Tue, Aug 8, 2017 at 1:17 PM, Lucian Ochian <okianl at yahoo.com>
> wrote:
> > >>
> > >>> Simon,
> > >>> Do you have a demo app with that? I am just curious to see a
> > >>> spring(boot) app with authorizations...I remember that I tried
> > something
> > >>> with authorizations, and the authorization context was null(I know
> > there
> > >>> are some Jira issues about it), but I still could not get it to work
> in
> > >>> 2.5.5
> > >>> AuthorizationContext authzContext =
> > >>>         keycloakSecurityContext.getAuthorizationContext();
> > >>> Thanks,Lucian
> > >>>
> > >>> On Tuesday, August 8, 2017, 10:25:35 AM CDT, Simon Payne <
> > >>> simonpayne58 at gmail.com> wrote:
> > >>>
> > >>> yes correct.
> > >>>
> > >>> there is a definite change in behavior with the addition of the
> > >>> keycloak.policy-enforcer-config.online-introspection=true  flag, as
> > >>> without
> > >>> this single line in my property file it works correctly as a bearer
> > only
> > >>> resource server.  Addition of this line results in the incorrect call
> > to
> > >>> token exchange endpoint.
> > >>>
> > >>> thanks
> > >>>
> > >>>
> > >>> On Tue, Aug 8, 2017 at 3:28 PM, Bill Burke <bburke at redhat.com>
> wrote:
> > >>>
> > >>> > Doesn't look like the switch is hooked up to anything.  As it is,
> it
> > >>> > looks like this switch was added for RPT validation, not access
> token
> > >>> > validation, and not ever implemented.  You just want the adapter to
> > >>> > validate the access token with the auth server for bearer token
> > >>> > requests, right?
> > >>> >
> > >>> >
> > >>> > On 8/8/17 9:29 AM, Bill Burke wrote:
> > >>> > > I'm looking at the code on server and I dont' see that it
> requires
> > >>> any
> > >>> > > special switch to use it.  The endpoint is:
> > >>> > >
> > >>> > > @Post
> > >>> > >
> > >>> > > /auth/realms/{realm}/protocol/openid-connect/token/introspect
> > >>> > >
> > >>> > > Takes form params.
> > >>> > >
> > >>> > > token
> > >>> > >
> > >>> > > token_type_hint (optional and defaults to "access_token")
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > On 8/8/17 4:31 AM, Simon Payne wrote:
> > >>> > >> after some debugging i figured that
> > >>> > >> keycloak.policy-enforcer-config.online-introspection=true
> > switched
> > >>> on
> > >>> > this
> > >>> > >> functionality, however it appears to error on a 400 after
> making a
> > >>> call
> > >>> > to
> > >>> > >> the /auth/realms/master/protocol/openid-connect/token endpoint.
> > >>> > >>
> > >>> > >> I'm assuming this is a bug?
> > >>> > >>
> > >>> > >> Thanks
> > >>> > >>
> > >>> > >>
> > >>> > >>
> > >>> > >> On Mon, Aug 7, 2017 at 3:10 PM, Simon Payne <
> > simonpayne58 at gmail.com
> > >>> >
> > >>> > wrote:
> > >>> > >>
> > >>> > >>> Hi All,
> > >>> > >>>
> > >>> > >>> I'm evaluating keycloak and i'm currently looking at token
> > >>> > introspection.
> > >>> > >>>
> > >>> > >>> I've managed to achieve this manually, i.e. by sending a post
> via
> > >>> > postman,
> > >>> > >>> but i'm unable to figure out whether this can be achieved via
> the
> > >>> > keycloak
> > >>> > >>> adapters, specifically spring boot.
> > >>> > >>>
> > >>> > >>> any help in this area would be appreciated.
> > >>> > >>>
> > >>> > >>> thanks
> > >>> > >>>
> > >>> > >>> Simon.
> > >>> > >>>
> > >>> > >> _______________________________________________
> > >>> > >> keycloak-user mailing list
> > >>> > >> keycloak-user at lists.jboss.org
> > >>> > >> https://lists.jboss.org/mailman/listinfo/keycloak-user
> > >>> > > _______________________________________________
> > >>> > > keycloak-user mailing list
> > >>> > > keycloak-user at lists.jboss.org
> > >>> > > https://lists.jboss.org/mailman/listinfo/keycloak-user
> > >>> >
> > >>> > _______________________________________________
> > >>> > keycloak-user mailing list
> > >>> > keycloak-user at lists.jboss.org
> > >>> > https://lists.jboss.org/mailman/listinfo/keycloak-user
> > >>> >
> > >>> _______________________________________________
> > >>> keycloak-user mailing list
> > >>> keycloak-user at lists.jboss.org
> > >>> https://lists.jboss.org/mailman/listinfo/keycloak-user
> > >>> _______________________________________________
> > >>> keycloak-user mailing list
> > >>> keycloak-user at lists.jboss.org
> > >>> https://lists.jboss.org/mailman/listinfo/keycloak-user
> > >>>
> > >>
> > >>
> > >
> > _______________________________________________
> > keycloak-user mailing list
> > keycloak-user at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-user
> >
>
>
> ------------------------------
>
> Message: 3
> Date: Thu, 10 Aug 2017 14:32:50 +0200
> From: Hynek Mlnarik <hmlnarik at redhat.com>
> Subject: Re: [keycloak-user] JGroups failure: failed submitting
>         DONT_BUNDLE message to thread pool
> To: Edwin de Jong <edwin.de.jong at simacan.com>
> Cc: keycloak-user <keycloak-user at lists.jboss.org>
> Message-ID:
>         <CAMvXD=HpizBoVGpVALHnmkwJHLFzh3U6tNwVbkTwi+Lwa-XTsw at mail.
> gmail.com>
> Content-Type: text/plain; charset="UTF-8"
>
> You seem to be facing https://issues.jboss.org/browse/WFLY-6179. Once
> keycloak updates to WF 10.1/11.x, this issue should be resolved.
>
> On Wed, Aug 9, 2017 at 1:32 PM, Edwin de Jong <edwin.de.jong at simacan.com>
> wrote:
> > Dear Keycloak users (and devs),
> >
> > This morning, we faced a production level issue on our Keycloak Cluster,
> > running in a 3-node formation on EC2. Symptoms were a high failure rate
> > of requests (> 20%) and high latency (> 10 seconds). We are currently
> > trying to figure out what went wrong. We would appreciate it if someone
> > with knowledge op JGroups / Inifinispan could chime in with a working
> > hypothesis.
> >
> > About priority: we are currently running nominally. We have brought down
> > two of the instances and brought up two new instances. The cluster is
> > working again as expected.
> >
> > Below I'll give information about our setup, the relevant log-messages
> > and links to some screenshots of our monitoring:
> >
> > EC2 instances are C4.Large (3x)
> > Keycloak Version 3.1.0-FINAL
> >
> > Normal CPU usage is around 1% or less. It peaked to 16% during the
> incident.
> > Memory usage is normal.
> >
> > Screenshots:
> >
> > - datadog statistics of our services calling keycloak:
> https://ibb.co/dsDTKv
> > - AWS EC2 instance Cloudwatch statistics: network out rate (in bytes per
> > MINUTE): https://ibb.co/j8jhCF
> > - AWS EC2 instance Cloudwatch statistics: network in rate (in bytes per
> > MINUTE): https://ibb.co/ggLuRa
> >
> > Log lines, just before failure (to help reduce clutter, I've removed the
> > date and replaced the IP addresses with "IP-A", "IP-B", "IP-C"). The
> > last message is repeated around 500.000 times in the span of around 1
> > minute.
> >
> > ---------------------->%-----------------------
> > 05:09:23,925 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-17,ee,ip-B) ISPN000094: Received new cluster view for channel
> > server: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,926 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-17,ee,ip-B) ISPN000094: Received new cluster view for channel
> > keycloak: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,926 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-17,ee,ip-B) ISPN000094: Received new cluster view for channel
> > web: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,926 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-17,ee,ip-B) ISPN000094: Received new cluster view for channel
> > ejb: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,928 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-17,ee,ip-B) ISPN000094: Received new cluster view for channel
> > hibernate: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,990 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-A) ISPN000094: Received new cluster view for channel
> > server: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,990 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-A) ISPN000094: Received new cluster view for channel
> > keycloak: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,991 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-A) ISPN000094: Received new cluster view for channel
> > web: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,992 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-A) ISPN000094: Received new cluster view for channel
> > hibernate: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,992 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-A) ISPN000094: Received new cluster view for channel
> > ejb: [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:23,996 INFO  [org.infinispan.CLUSTER] (transport-thread--p14-t18)
> > ISPN000310: Starting cluster-wide rebalance for cache authorization,
> > topology CacheTopology{id=6, rebalanceId=3,
> > currentCH=DefaultConsistentHash{ns=80, owners = (2)[ip-A: 54+26, ip-B:
> > 26+54]}, pendingCH=DefaultConsistentHash{ns=80, owners = (2)[ip-A:
> > 40+40, ip-B: 40+40]}, unionCH=null, actualMembers=[ip-A, ip-B]}
> >  05:09:24,001 INFO  [org.infinispan.CLUSTER] (transport-thread--p14-t18)
> > ISPN000310: Starting cluster-wide rebalance for cache sessions, topology
> > CacheTopology{id=6, rebalanceId=3,
> > currentCH=DefaultConsistentHash{ns=80, owners = (2)[ip-A: 54+26, ip-B:
> > 26+54]}, pendingCH=DefaultConsistentHash{ns=80, owners = (2)[ip-A:
> > 40+40, ip-B: 40+40]}, unionCH=null, actualMembers=[ip-A, ip-B]}
> >  05:09:24,004 INFO  [org.infinispan.CLUSTER] (transport-thread--p14-t18)
> > ISPN000310: Starting cluster-wide rebalance for cache offlineSessions,
> > topology CacheTopology{id=6, rebalanceId=3,
> > currentCH=DefaultConsistentHash{ns=80, owners = (2)[ip-A: 54+26, ip-B:
> > 26+54]}, pendingCH=DefaultConsistentHash{ns=80, owners = (2)[ip-A:
> > 40+40, ip-B: 40+40]}, unionCH=null, actualMembers=[ip-A, ip-B]}
> >  05:09:24,014 INFO  [org.infinispan.CLUSTER] (transport-thread--p14-t18)
> > ISPN000310: Starting cluster-wide rebalance for cache loginFailures,
> > topology CacheTopology{id=6, rebalanceId=3,
> > currentCH=DefaultConsistentHash{ns=80, owners = (2)[ip-A: 54+26, ip-B:
> > 26+54]}, pendingCH=DefaultConsistentHash{ns=80, owners = (2)[ip-A:
> > 40+40, ip-B: 40+40]}, unionCH=null, actualMembers=[ip-A, ip-B]}
> >  05:09:24,027 INFO  [org.infinispan.CLUSTER] (remote-thread--p7-t130)
> > ISPN000336: Finished cluster-wide rebalance for cache sessions, topology
> > id = 6
> >  05:09:24,028 INFO  [org.infinispan.CLUSTER] (remote-thread--p7-t130)
> > ISPN000336: Finished cluster-wide rebalance for cache offlineSessions,
> > topology id = 6
> >  05:09:24,029 INFO  [org.infinispan.CLUSTER] (remote-thread--p7-t131)
> > ISPN000336: Finished cluster-wide rebalance for cache loginFailures,
> > topology id = 6
> >  05:09:24,029 INFO  [org.infinispan.CLUSTER] (remote-thread--p7-t132)
> > ISPN000336: Finished cluster-wide rebalance for cache authorization,
> > topology id = 6
> >  05:09:33,567 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-C) ISPN000093: Received new, MERGED cluster view for
> > channel server: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2 subgroups:
> > [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,569 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-C) ISPN000093: Received new, MERGED cluster view for
> > channel keycloak: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2
> > subgroups: [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,569 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-C) ISPN000093: Received new, MERGED cluster view for
> > channel web: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2 subgroups:
> > [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,573 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-C) ISPN000093: Received new, MERGED cluster view for
> > channel ejb: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2 subgroups:
> > [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,575 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-1,ee,ip-C) ISPN000093: Received new, MERGED cluster view for
> > channel hibernate: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2
> > subgroups: [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,521 WARN  [org.jgroups.protocols.pbcast.NAKACK]
> > (Incoming-19,ee,ip-B) JGRP000011: ip-B: dropped message 54375 from
> > non-member ip-C (view=[ip-A|3] (2) [ip-A, ip-B]) Warning
> >  05:09:33,527 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-20,ee,ip-B) ISPN000093: Received new, MERGED cluster view for
> > channel server: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2 subgroups:
> > [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,529 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-20,ee,ip-B) ISPN000093: Received new, MERGED cluster view for
> > channel keycloak: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2
> > subgroups: [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,529 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-20,ee,ip-B) ISPN000093: Received new, MERGED cluster view for
> > channel web: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2 subgroups:
> > [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,530 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-20,ee,ip-B) ISPN000093: Received new, MERGED cluster view for
> > channel ejb: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2 subgroups:
> > [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,533 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-20,ee,ip-B) ISPN000093: Received new, MERGED cluster view for
> > channel hibernate: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2
> > subgroups: [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,518 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-7,ee,ip-A) ISPN000093: Received new, MERGED cluster view for
> > channel server: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2 subgroups:
> > [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,525 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-7,ee,ip-A) ISPN000093: Received new, MERGED cluster view for
> > channel keycloak: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2
> > subgroups: [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,525 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-7,ee,ip-A) ISPN000093: Received new, MERGED cluster view for
> > channel web: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2 subgroups:
> > [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,527 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-7,ee,ip-A) ISPN000093: Received new, MERGED cluster view for
> > channel hibernate: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2
> > subgroups: [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,527 INFO
> > [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> > (Incoming-7,ee,ip-A) ISPN000093: Received new, MERGED cluster view for
> > channel ejb: MergeView::[ip-A|4] (3) [ip-A, ip-B, ip-C], 2 subgroups:
> > [ip-A|2] (3) [ip-A, ip-C, ip-B], [ip-A|3] (2) [ip-A, ip-B]
> >  05:09:33,577 INFO  [org.infinispan.CLUSTER] (transport-thread--p14-t12)
> > ISPN000310: Starting cluster-wide rebalance for cache authorization,
> > topology CacheTopology{id=10, rebalanceId=3,
> > currentCH=DefaultConsistentHash{ns=80, owners = (3)[ip-A: 27+53, ip-C:
> > 27+53, ip-B: 26+54]}, pendingCH=DefaultConsistentHash{ns=80, owners =
> > (3)[ip-A: 27+53, ip-B: 26+54, ip-C: 27+53]}, unionCH=null,
> > actualMembers=[ip-A, ip-B, ip-C]}
> >  05:09:33,579 INFO  [org.infinispan.CLUSTER] (transport-thread--p14-t12)
> > ISPN000310: Starting cluster-wide rebalance for cache offlineSessions,
> > topology CacheTopology{id=10, rebalanceId=3,
> > currentCH=DefaultConsistentHash{ns=80, owners = (3)[ip-A: 27+53, ip-C:
> > 27+53, ip-B: 26+54]}, pendingCH=DefaultConsistentHash{ns=80, owners =
> > (3)[ip-A: 27+53, ip-B: 26+54, ip-C: 27+53]}, unionCH=null,
> > actualMembers=[ip-A, ip-B, ip-C]}
> >  05:09:33,580 INFO  [org.infinispan.CLUSTER] (transport-thread--p14-t12)
> > ISPN000310: Starting cluster-wide rebalance for cache loginFailures,
> > topology CacheTopology{id=10, rebalanceId=3,
> > currentCH=DefaultConsistentHash{ns=80, owners = (3)[ip-A: 27+53, ip-C:
> > 27+53, ip-B: 26+54]}, pendingCH=DefaultConsistentHash{ns=80, owners =
> > (3)[ip-A: 27+53, ip-B: 26+54, ip-C: 27+53]}, unionCH=null,
> > actualMembers=[ip-A, ip-B, ip-C]}
> >  05:09:33,582 INFO  [org.infinispan.CLUSTER] (transport-thread--p14-t15)
> > ISPN000310: Starting cluster-wide rebalance for cache sessions, topology
> > CacheTopology{id=10, rebalanceId=3,
> > currentCH=DefaultConsistentHash{ns=80, owners = (3)[ip-A: 27+53, ip-C:
> > 27+53, ip-B: 26+54]}, pendingCH=DefaultConsistentHash{ns=80, owners =
> > (3)[ip-A: 27+53, ip-B: 26+54, ip-C: 27+53]}, unionCH=null,
> > actualMembers=[ip-A, ip-B, ip-C]}
> >  05:09:33,589 INFO  [org.infinispan.CLUSTER] (remote-thread--p7-t130)
> > ISPN000336: Finished cluster-wide rebalance for cache loginFailures,
> > topology id = 10
> >  05:09:33,589 INFO  [org.infinispan.CLUSTER] (remote-thread--p7-t130)
> > ISPN000336: Finished cluster-wide rebalance for cache authorization,
> > topology id = 10
> >  05:09:33,590 INFO  [org.infinispan.CLUSTER] (remote-thread--p7-t131)
> > ISPN000336: Finished cluster-wide rebalance for cache offlineSessions,
> > topology id = 10
> >  05:09:33,601 INFO  [org.infinispan.CLUSTER] (remote-thread--p7-t133)
> > ISPN000336: Finished cluster-wide rebalance for cache sessions, topology
> > id = 10
> >  05:09:34,001 ERROR [org.jgroups.protocols.TCP] (Connection.Receiver
> > [IP-A:35361 - IP-C:7600],ee,ip-A) ip-A: failed submitting DONT_BUNDLE
> > message to thread pool: java.util.concurrent.RejectedExecutionException:
> > Task org.jgroups.protocols.TP$SingleMessageHandler at 17c86a0c rejected
> > from java.util.concurrent.ThreadPoolExecutor at 52cae691[Running, pool size
> > = 4, active threads = 4, queued tasks = 100, completed tasks = 1589693].
> > Msg: RequestCorrelator: id=200, type=REQ, id=607, rsp_expected=true,
> > FORK: ee:keycloak, NAKACK: [XMIT_RSP, seqno=206], TCP: [cluster_name=ee]
> > ERROR
> >  05:09:34,010 ERROR [org.jgroups.protocols.TCP] (Connection.Receiver
> > [IP-A:35361 - IP-C:7600],ee,ip-A) ip-A: failed submitting DONT_BUNDLE
> > message to thread pool: java.util.concurrent.RejectedExecutionException:
> > Task org.jgroups.protocols.TP$SingleMessageHandler at 4fec3655 rejected
> > from java.util.concurrent.ThreadPoolExecutor at 52cae691[Running, pool size
> > = 4, active threads = 4, queued tasks = 100, completed tasks = 1589694].
> > Msg: RequestCorrelator: id=200, type=REQ, id=609, rsp_expected=true,
> > FORK: ee:keycloak, NAKACK: [XMIT_RSP, seqno=208], TCP: [cluster_name=ee]
> > ERROR
> >
> > (last line repeated many, many times)
> > ---------------------->%-----------------------
> >
> >
> >
> > Infinispan subsystem configuration in standalone-ha.xml:
> >
> > ---------------------->%-----------------------
> >     <subsystem xmlns="urn:jboss:domain:infinispan:4.0">
> >       <cache-container name="keycloak" jndi-name="infinispan/Keycloak">
> >         <transport lock-timeout="60000"/>
> >         <local-cache name="realms">
> >           <eviction max-entries="10000" strategy="LRU"/>
> >         </local-cache>
> >         <local-cache name="users">
> >           <eviction max-entries="10000" strategy="LRU"/>
> >         </local-cache>
> >         <distributed-cache name="sessions" mode="SYNC" owners="3"/>
> >         <distributed-cache name="offlineSessions" mode="SYNC"
> owners="3"/>
> >         <distributed-cache name="loginFailures" mode="SYNC" owners="3"/>
> >         <distributed-cache name="authorization" mode="SYNC" owners="3"/>
> >         <replicated-cache name="work" mode="SYNC"/>
> >         <local-cache name="keys">
> >           <eviction max-entries="1000" strategy="LRU"/>
> >           <expiration max-idle="3600000"/>
> >         </local-cache>
> >       </cache-container>
> >       <cache-container name="server" aliases="singleton cluster"
> > default-cache="default" module="org.wildfly.clustering.server">
> >         <transport lock-timeout="60000"/>
> >         <replicated-cache name="default" mode="SYNC">
> >           <transaction mode="BATCH"/>
> >         </replicated-cache>
> >       </cache-container>
> >       <cache-container name="web" default-cache="dist"
> > module="org.wildfly.clustering.web.infinispan">
> >         <transport lock-timeout="60000"/>
> >         <distributed-cache name="dist" mode="ASYNC" l1-lifespan="0"
> > owners="2">
> >           <locking isolation="REPEATABLE_READ"/>
> >           <transaction mode="BATCH"/>
> >           <file-store/>
> >         </distributed-cache>
> >       </cache-container>
> >       <cache-container name="ejb" aliases="sfsb" default-cache="dist"
> > module="org.wildfly.clustering.ejb.infinispan">
> >         <transport lock-timeout="60000"/>
> >         <distributed-cache name="dist" mode="ASYNC" l1-lifespan="0"
> > owners="2">
> >           <locking isolation="REPEATABLE_READ"/>
> >           <transaction mode="BATCH"/>
> >           <file-store/>
> >         </distributed-cache>
> >       </cache-container>
> >       <cache-container name="hibernate" default-cache="local-query"
> > module="org.hibernate.infinispan">
> >         <transport lock-timeout="60000"/>
> >         <local-cache name="local-query">
> >           <eviction strategy="LRU" max-entries="10000"/>
> >           <expiration max-idle="100000"/>
> >         </local-cache>
> >         <invalidation-cache name="entity" mode="SYNC">
> >           <transaction mode="NON_XA"/>
> >           <eviction strategy="LRU" max-entries="10000"/>
> >           <expiration max-idle="100000"/>
> >         </invalidation-cache>
> >         <replicated-cache name="timestamps" mode="ASYNC"/>
> >       </cache-container>
> >     </subsystem>
> > ---------------------->%-----------------------
> >
> > JGroups subsystem configuration:
> >
> > ---------------------->%-----------------------
> >     <subsystem xmlns="urn:jboss:domain:jgroups:4.0" default-stack="tcp">
> >       <channels default="ee">
> >         <channel name="ee" stack="tcp"/>
> >       </channels>
> >       <stacks>
> >         <stack name="tcp">
> >           <transport type="TCP" socket-binding="jgroups-tcp"/>
> >           <protocol type="S3_PING">
> >             <property name="access_key">S3AccessKey</property>
> >             <property name="secret_access_key">
> S3SecretAccessKey</property>
> >             <property name="location">S3PingBucketName</property>
> >           </protocol>
> >           <protocol type="MERGE2"/>
> >           <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
> >           <protocol type="FD"/>
> >           <protocol type="VERIFY_SUSPECT"/>
> >           <protocol type="pbcast.NAKACK"/>
> >           <protocol type="UNICAST2"/>
> >           <protocol type="pbcast.STABLE"/>
> >           <protocol type="pbcast.GMS"/>
> >           <protocol type="UFC"/>
> >           <protocol type="MFC"/>
> >           <protocol type="FRAG2"/>
> >           <protocol type="RSVP"/>
> >         </stack>
> >         <stack name="udp">
> >           <transport type="UDP" socket-binding="jgroups-udp"/>
> >           <protocol type="PING"/>
> >           <protocol type="MERGE3"/>
> >           <protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
> >           <protocol type="FD_ALL"/>
> >           <protocol type="VERIFY_SUSPECT"/>
> >           <protocol type="pbcast.NAKACK2"/>
> >           <protocol type="UNICAST3"/>
> >           <protocol type="pbcast.STABLE"/>
> >           <protocol type="pbcast.GMS"/>
> >           <protocol type="UFC"/>
> >           <protocol type="MFC"/>
> >           <protocol type="FRAG2"/>
> >         </stack>
> >       </stacks>
> >     </subsystem>
> > ---------------------->%-----------------------
> >
> > with kind regards,
> >
> > Edwin de Jong
> >
> > -- Simacan B.V. Data Engineer
> > _______________________________________________
> > keycloak-user mailing list
> > keycloak-user at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-user
>
>
>
> --
>
> --Hynek
>
>
> ------------------------------
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>
> End of keycloak-user Digest, Vol 44, Issue 23
> *********************************************
>