[keycloak-user] Sync Issues
Aaron Echols
aechols at bfcsaz.com
Wed Jul 18 15:25:24 EDT 2018
Hi Dmitry,
I did as you suggested, but something seems amiss. When looking under:
MBeans > org.wildfly.clustering.infinispan > CacheManager > "keycloak" >
CacheManager > Attributes > clusterMembers
shows the same hosts 2x: [srv-iam-01, srv-iam-01], the later should be 02.
The other option you said to look it didn't seem to actually exist:
MBeans -> org.wildfly.clustering.infinispan -> Cache -> "keycloak" -> Cache
I'm still confused and looking through the configs to see if I can figure
out what is going on. Thanks :)
--
*Aaron Echols*
Lead Administrator (IT)
Benjamin Franklin Charter School | IT
Email: aechols at bfcsaz.com
Phone: (480) 677-8400
Website: http://www.bfcsaz.com
Support Email: techsupport at bfcsaz.com
Support Portal: https://bfcs.freshservice.com/support/home
Common Questions: https://bfcs.freshservice.com/support/solutions
Forgot your password: https://accounts.bfcsaz.com
<https://www.facebook.com/bfcsaz/> <https://twitter.com/bfcs_k12>
<https://www.instagram.com/bfcs_k12>
*CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is
for the sole use of the intended recipient(s) and may contain confidential
and privileged information. Any unauthorized review, copy, use, disclosure,
or distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply e-mail and destroy all copies of the
original message.
On Tue, Jul 17, 2018 at 4:01 PM Aaron Echols <aechols at bfcsaz.com> wrote:
> Hi Dmitry,
>
> Thanks for the reply!
>
> I just finished upgrading to 4.1.0 and the issue persists...
>
> Let me try running the console and take a look there and see what it
> shows. I'll post back shortly. Thanks for the help!
> --
> *Aaron Echols*
>
> On Tue, Jul 17, 2018 at 3:58 PM Dmitry Telegin <dt at acutus.pro> wrote:
>
>> Hi Aaron,
>>
>> This all sounds very weird. Off the top of my head:
>> - try latest Keycloak (4.1.0), is the issue reproducible?
>> - Infinispan exposes quite a lot of stuff via JMX. Run JMC or JConsole,
>> connect to the Keycloak process, go to MBeans ->
>> org.wildfly.clustering.infinispan -> Cache -> "keycloak" -> Cache. How
>> many caches are there? (should be 15 as of KC 4.1.0) Are they all
>> running? Are there any abnormalities? Entries under CacheManager might
>> be useful, too.
>>
>> Cheers,
>> Dmitry Telegin
>> CTO, Acutus s.r.o.
>> Keycloak Consulting and Training
>>
>> Pod lipami street 339/52, 130 00 Prague 3, Czech Republic
>> +42 (022) 888-30-71
>> E-mail: info at acutus.pro
>>
>> On Tue, 2018-07-17 at 13:28 -0700, Aaron Echols wrote:
>> > Hello All,
>> >
>> > I've successfully setup a cluster with 2 nodes. Everything is working
>> > great, except for one issue I can't figure out. I'm starting to pull my
>> > hair out and wanted to see if anyone else has seen the issue and how to
>> > correct it.
>> >
>> > I've setup a user federation using Active Directory (Server 2016) using
>> > Keycloak 3.4.3. They are load balanced behind Netscaler 12.0.x.
>> Infinispan
>> > seems to be working correctly. It's backed by a MariaDB 10.1.x, 3 node
>> > cluster. Things I've noted:
>> >
>> > - I can create a local user and it syncs instantly between the KC
>> 3.4.3
>> > nodes
>> > - Password syncs work, all changes to attributes sync, etc
>> > - I change settings for the user federation I created and they DON'T
>> > sync, so creating a mapper, changing a sync setting, etc, they have
>> to be
>> > changed by hand manually on each node.
>> > - Same with Role and realm-management. I can apply a permission to a
>> > group or user and it doesn't sync.
>> > - If I restart the wildfly server, the changes to propagate to the
>> > opposite node everytime.
>> >
>> >
>> >
>> > I deleted a custom role in the realm-management client, and it deleted
>> it
>> > from the database. On the secondary node, I saw the file was still
>> listed,
>> > even with hard refreshes of the browser. I clicked to delete the custom
>> > role and got the following in the server.log:
>> >
>> >
>> >
>> > ERROR [org.keycloak.services.error.KeycloakErrorHandler] (default
>> task-26)
>> > Uncaught server error: java.lang.IllegalStateException: Not found in
>> > database
>> > at
>> >
>> org.keycloak.models.cache.infinispan.RoleAdapter.isUpdated(RoleAdapter.java:66)
>> > at
>> >
>> org.keycloak.models.cache.infinispan.RoleAdapter.getId(RoleAdapter.java:105)
>> > at
>> >
>> org.keycloak.models.cache.infinispan.RealmCacheSession.removeRole(RealmCacheSession.java:736)
>> > at
>> >
>> org.keycloak.models.cache.infinispan.ClientAdapter.removeRole(ClientAdapter.java:587)
>> > at
>> >
>> org.keycloak.services.resources.admin.RoleResource.deleteRole(RoleResource.java:53)
>> > at
>> >
>> org.keycloak.services.resources.admin.RoleByIdResource.deleteRole(RoleByIdResource.java:115)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > at java.lang.reflect.Method.invoke(Method.java:498)
>> > at
>> >
>> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140)
>> > at
>> >
>> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)
>> > at
>> >
>> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)
>> > at
>> >
>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:138)
>> > at
>> >
>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:107)
>> > at
>> >
>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:133)
>> > at
>> >
>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:107)
>> > at
>> >
>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:133)
>> > at
>> >
>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:101)
>> > at
>> >
>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406)
>> > at
>> >
>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:213)
>> > at
>> >
>> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:228)
>> > at
>> >
>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
>> > at
>> >
>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>> > at
>> >
>> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
>> > at
>> >
>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
>> > at
>> >
>> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:90)
>> > at
>> > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>> > at
>> >
>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>> > at
>> >
>> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
>> > at
>> >
>> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
>> > at
>> >
>> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
>> > at
>> >
>> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
>> > at
>> >
>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>> > at
>> >
>> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
>> > at
>> >
>> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
>> > at
>> >
>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>> > at
>> >
>> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
>> > at
>> >
>> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
>> > at
>> >
>> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
>> > at
>> >
>> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
>> > at
>> >
>> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
>> > at
>> >
>> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
>> > at
>> >
>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>> > at
>> >
>> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
>> > at
>> >
>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>> > at
>> >
>> org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68)
>> > at
>> >
>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>> > at
>> >
>> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
>> > at
>> >
>> io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
>> > at
>> >
>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
>> > at
>> >
>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
>> > at
>> >
>> io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
>> > at
>> >
>> io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
>> > at
>> >
>> org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
>> > at
>> >
>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
>> > at
>> >
>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
>> > at
>> >
>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
>> > at
>> >
>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
>> > at
>> >
>> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
>> > at
>> >
>> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
>> > at
>> >
>> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
>> > at
>> > io.undertow.server.Connectors.executeRootHandler(Connectors.java:326)
>> > at
>> > io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> > at java.lang.Thread.run(Thread.java:748)
>> >
>> >
>> >
>> > I'm not sure if there is an issue with Infinispan or a sql connection
>> > issue. I've included my SQL connection string as well:
>> >
>> >
>> >
>> > <datasource
>> jndi-name="java:jboss/datasources/KeycloakDS"
>> > pool-name="KeycloakDS" enabled="true" use-java-context="true">
>> > <connection-url>jdbc:mariadb://
>> >
>> 10.5.30.202:3306/keycloak?useUnicode=yes;characterEncoding=UTF-8;sessionVariables=wait_timeout=180;autoRe
>> > connect=true</connection-url>
>> > <driver>mariadb</driver>
>> > <pool>
>> > <max-pool-size>20</max-pool-size>
>> > </pool>
>> > <security>
>> > <user-name>keycloak_user</user-name>
>> > <password><some-passphrase></password>
>> > </security>
>> > <validation>
>> > <check-valid-connection-sql>select
>> > 1</check-valid-connection-sql>
>> > <validate-on-match>true</validate-on-match>
>> >
>> <background-validation>true</background-validation>
>> >
>> > <background-validation-millis>10000</background-validation-millis>
>> > </validation>
>> > </datasource>
>> > <drivers>
>> > <!-- driver declaration -->
>> > <driver name="mariadb" module="org.mariadb">
>> >
>> > <xa-datasource-class>org.mariadb.jdbc.Driver</xa-datasource-class>
>> > </driver>
>> > <driver name="h2" module="com.h2database.h2">
>> >
>> > <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-datasource-class>
>> > </driver>
>> > </drivers>
>> > </datasources>
>> >
>> >
>> >
>> > I'm using the mariadb-java-client-2.2.3 driver.
>> >
>> >
>> >
>> > <?xml version="1.0" ?>
>> > <module xmlns="urn:jboss:module:1.3" name="org.mariadb">
>> >
>> > <resources>
>> > <resource-root path="mariadb-java-client-2.2.3.jar"/>
>> > </resources>
>> >
>> > <dependencies>
>> > <module name="javax.api"/>
>> > <module name="javax.transaction.api"/>
>> > </dependencies>
>> > </module>
>> >
>> >
>> > Any assistance would be appreciated. I'll grab whatever information is
>> > needed. Thank you in advance. :)
>> > --
>> > *Aaron Echols*
>> > _______________________________________________
>> > keycloak-user mailing list
>> > keycloak-user at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>
More information about the keycloak-user
mailing list