8:07 a.m.
On Tue, Aug 7, 2018 at 3:28 PM Hayden Fuss <hfuss(a)bandwidth.com> wrote:
Hello,
We are attempting to run Keycloak on two OpenShift clusters using remote
ISPNs and a single MariaDB instance. We're hacking together the Keycloak on
Openshift blogpost, the JDG-as-a-service demo from Summit, RH SSO demo from
Summit, and following the Keycloak/RH SSO basic setup guide to Cross-DC
replication. The hope is do an initial evaluation of Keycloak's
availability.
We were able to create a new user on master (site1), disable the user on
master2 (site2), and see the user was disabled on master. So ISPN
replication seems to be working because the work cache was replicated to
invalidate the local caches. However, the sessions cache does not seem to
be replicated because when logged in as the same user on the two different
Keycloaks (in Incognito mode) there is only one active session shown on
both UIs and the timestamp/IP/etc is different for the listed session.
So at this point the Infinispan cluster within a single DC works correctly
[1] (the one that is formed by KUBE_PING). The Cross-DC cluster (also known
as the Global Cluster) also works correctly [2]. Users cache replicates
fine but sessions don't.
If I understood everything correctly, there might be two issues there.
The first one is Infinispan misconfiguration (I briefly looked through the
configuration and can not spot any mistake but there might be some typo or
anything like that). That one is easy to be verified, just put an entry on
one node (e.g. using REST [3]) and see if it's available on the other one
(again, using REST for example [4]).
If this test works fine, you can check if Keycloak forwards traffic to the
Infinispan cluster. The easiest way is to set a breakpoint somewhere
in
org.keycloak.models.sessions.infinispan.changes.sessions.LastSessionRefreshChecker#shouldSaveClientSessionToRemoteCache
and
org.keycloak.models.sessions.infinispan.changes.sessions.LastSessionRefreshChecker#shouldSaveUserSessionToRemoteCache.
[1] can be verified by calling `oc logs infinispan-app | grep view`
[2] can be verified by calling `oc logs infinispan-app | grep "x-site"`
[3] curl -d test ISPN_IP:8080/rest/sessions/test
[4] curl ISPN_IP2:8080/rest/sessions/test
We are using the latest, stable Keycloak image, version 4.1.0.Final,
and
the latest, stable Infinispan image for to act as our data grid, version
9.3.1.Final, which we know differs from the 8.2.8 version Keycloak uses for
its local caches.
We were trying one Keycloak node and two ISPN nodes in each cluster, but
for simplicity we've attached logs where we only ran one Keycloak and one
ISPN in each cluster.
We were connecting to the two different Keycloaks via two different
OpenShift Routes without a load balancer to fake sticky sessions for now.
Keycloak connects to ISPN via a "HotRod" Service. ISPN connects to other
nodes within the same cluster via KUBE_PING, and discovers the other
cluster via TCPPING hitting a particular OpenShift app node from that
cluster that exposes the "discovery" Service with a NodePort. The Keycloaks
share the single MariaDB through a NodePort Service in one of the clusters
as well.
The logs didn't seem to contain any of the messages in the trouble shooting
guide. We had trouble using JMX to check the ISPNs because they were
running in containers, but we've using the CLI tool and the Infinispan
management console to try to troubleshoot but any key we pulled from the
logs that we thought was a session ID was not in the caches and we could
not find a way to simply list all keys in the caches.
Below is a viewable link to a zip containing logs from the scenario
described in the second paragraph, and our config files.
https://drive.google.com/open?id=0B_OCdNCEtoCYOU12T3dEUFplS193VFNFbEFYclB...
Thanks for your time and help!
Best,
Hayden
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-user