[keycloak-user] Sync Issues
Aaron Echols
aechols at bfcsaz.com
Wed Jul 18 15:32:16 EDT 2018
Ok, I fixed a variable in my
/etc/default/wildfly.conf
Forgot to change the hostname in there:
# Hostname:
WILDFLY_HOST=srv-iam-02
Once I fixed that, the server started syncing immediately. Thanks for
helping point me in the right direction. :)
--
*Aaron Echols*
On Wed, Jul 18, 2018 at 12:25 PM Aaron Echols <aechols at bfcsaz.com> wrote:
> Hi Dmitry,
>
> I did as you suggested, but something seems amiss. When looking under:
>
> MBeans > org.wildfly.clustering.infinispan > CacheManager > "keycloak" >
> CacheManager > Attributes > clusterMembers
>
> shows the same hosts 2x: [srv-iam-01, srv-iam-01], the later should be 02.
> The other option you said to look it didn't seem to actually exist:
>
> MBeans -> org.wildfly.clustering.infinispan -> Cache -> "keycloak" -> Cache
>
> I'm still confused and looking through the configs to see if I can figure
> out what is going on. Thanks :)
> --
> *Aaron Echols*
> Lead Administrator (IT)
> Benjamin Franklin Charter School | IT
> Email: aechols at bfcsaz.com
> Phone: (480) 677-8400
> Website: http://www.bfcsaz.com
> Support Email: techsupport at bfcsaz.com
> Support Portal: https://bfcs.freshservice.com/support/home
> Common Questions: https://bfcs.freshservice.com/support/solutions
> Forgot your password: https://accounts.bfcsaz.com
>
> <https://www.facebook.com/bfcsaz/> <https://twitter.com/bfcs_k12>
> <https://www.instagram.com/bfcs_k12>
>
>
> *CONFIDENTIALITY NOTICE: This e-mail message, including any attachments,
> is for the sole use of the intended recipient(s) and may contain
> confidential and privileged information. Any unauthorized review, copy,
> use, disclosure, or distribution is prohibited. If you are not the intended
> recipient, please contact the sender by reply e-mail and destroy all copies
> of the original message.
>
>
> On Tue, Jul 17, 2018 at 4:01 PM Aaron Echols <aechols at bfcsaz.com> wrote:
>
>> Hi Dmitry,
>>
>> Thanks for the reply!
>>
>> I just finished upgrading to 4.1.0 and the issue persists...
>>
>> Let me try running the console and take a look there and see what it
>> shows. I'll post back shortly. Thanks for the help!
>> --
>> *Aaron Echols*
>>
>> On Tue, Jul 17, 2018 at 3:58 PM Dmitry Telegin <dt at acutus.pro> wrote:
>>
>>> Hi Aaron,
>>>
>>> This all sounds very weird. Off the top of my head:
>>> - try latest Keycloak (4.1.0), is the issue reproducible?
>>> - Infinispan exposes quite a lot of stuff via JMX. Run JMC or JConsole,
>>> connect to the Keycloak process, go to MBeans ->
>>> org.wildfly.clustering.infinispan -> Cache -> "keycloak" -> Cache. How
>>> many caches are there? (should be 15 as of KC 4.1.0) Are they all
>>> running? Are there any abnormalities? Entries under CacheManager might
>>> be useful, too.
>>>
>>> Cheers,
>>> Dmitry Telegin
>>> CTO, Acutus s.r.o.
>>> Keycloak Consulting and Training
>>>
>>> Pod lipami street 339/52, 130 00 Prague 3, Czech Republic
>>> +42 (022) 888-30-71
>>> E-mail: info at acutus.pro
>>>
>>> On Tue, 2018-07-17 at 13:28 -0700, Aaron Echols wrote:
>>> > Hello All,
>>> >
>>> > I've successfully setup a cluster with 2 nodes. Everything is working
>>> > great, except for one issue I can't figure out. I'm starting to pull my
>>> > hair out and wanted to see if anyone else has seen the issue and how to
>>> > correct it.
>>> >
>>> > I've setup a user federation using Active Directory (Server 2016) using
>>> > Keycloak 3.4.3. They are load balanced behind Netscaler 12.0.x.
>>> Infinispan
>>> > seems to be working correctly. It's backed by a MariaDB 10.1.x, 3 node
>>> > cluster. Things I've noted:
>>> >
>>> > - I can create a local user and it syncs instantly between the KC
>>> 3.4.3
>>> > nodes
>>> > - Password syncs work, all changes to attributes sync, etc
>>> > - I change settings for the user federation I created and they DON'T
>>> > sync, so creating a mapper, changing a sync setting, etc, they have
>>> to be
>>> > changed by hand manually on each node.
>>> > - Same with Role and realm-management. I can apply a permission to a
>>> > group or user and it doesn't sync.
>>> > - If I restart the wildfly server, the changes to propagate to the
>>> > opposite node everytime.
>>> >
>>> >
>>> >
>>> > I deleted a custom role in the realm-management client, and it deleted
>>> it
>>> > from the database. On the secondary node, I saw the file was still
>>> listed,
>>> > even with hard refreshes of the browser. I clicked to delete the custom
>>> > role and got the following in the server.log:
>>> >
>>> >
>>> >
>>> > ERROR [org.keycloak.services.error.KeycloakErrorHandler] (default
>>> task-26)
>>> > Uncaught server error: java.lang.IllegalStateException: Not found in
>>> > database
>>> > at
>>> >
>>> org.keycloak.models.cache.infinispan.RoleAdapter.isUpdated(RoleAdapter.java:66)
>>> > at
>>> >
>>> org.keycloak.models.cache.infinispan.RoleAdapter.getId(RoleAdapter.java:105)
>>> > at
>>> >
>>> org.keycloak.models.cache.infinispan.RealmCacheSession.removeRole(RealmCacheSession.java:736)
>>> > at
>>> >
>>> org.keycloak.models.cache.infinispan.ClientAdapter.removeRole(ClientAdapter.java:587)
>>> > at
>>> >
>>> org.keycloak.services.resources.admin.RoleResource.deleteRole(RoleResource.java:53)
>>> > at
>>> >
>>> org.keycloak.services.resources.admin.RoleByIdResource.deleteRole(RoleByIdResource.java:115)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> > at
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> > at
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> > at java.lang.reflect.Method.invoke(Method.java:498)
>>> > at
>>> >
>>> org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140)
>>> > at
>>> >
>>> org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)
>>> > at
>>> >
>>> org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)
>>> > at
>>> >
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:138)
>>> > at
>>> >
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:107)
>>> > at
>>> >
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:133)
>>> > at
>>> >
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:107)
>>> > at
>>> >
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:133)
>>> > at
>>> >
>>> org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:101)
>>> > at
>>> >
>>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406)
>>> > at
>>> >
>>> org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:213)
>>> > at
>>> >
>>> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:228)
>>> > at
>>> >
>>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
>>> > at
>>> >
>>> org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
>>> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
>>> > at
>>> >
>>> org.keycloak.services.filters.KeycloakSessionServletFilter.doFilter(KeycloakSessionServletFilter.java:90)
>>> > at
>>> > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
>>> > at
>>> >
>>> org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
>>> > at
>>> >
>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
>>> > at
>>> >
>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>> > at
>>> >
>>> io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
>>> > at
>>> >
>>> io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
>>> > at
>>> >
>>> io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
>>> > at
>>> >
>>> io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
>>> > at
>>> >
>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>> > at
>>> >
>>> org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
>>> > at
>>> >
>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>> > at
>>> >
>>> org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68)
>>> > at
>>> >
>>> io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
>>> > at
>>> >
>>> io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
>>> > at
>>> >
>>> io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
>>> > at
>>> >
>>> org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
>>> > at
>>> >
>>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
>>> > at
>>> >
>>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
>>> > at
>>> >
>>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
>>> > at
>>> >
>>> org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1508)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
>>> > at
>>> >
>>> io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
>>> > at
>>> > io.undertow.server.Connectors.executeRootHandler(Connectors.java:326)
>>> > at
>>> >
>>> io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:812)
>>> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> > at
>>> >
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> > at java.lang.Thread.run(Thread.java:748)
>>> >
>>> >
>>> >
>>> > I'm not sure if there is an issue with Infinispan or a sql connection
>>> > issue. I've included my SQL connection string as well:
>>> >
>>> >
>>> >
>>> > <datasource
>>> jndi-name="java:jboss/datasources/KeycloakDS"
>>> > pool-name="KeycloakDS" enabled="true" use-java-context="true">
>>> > <connection-url>jdbc:mariadb://
>>> >
>>> 10.5.30.202:3306/keycloak?useUnicode=yes;characterEncoding=UTF-8;sessionVariables=wait_timeout=180;autoRe
>>> > connect=true</connection-url>
>>> > <driver>mariadb</driver>
>>> > <pool>
>>> > <max-pool-size>20</max-pool-size>
>>> > </pool>
>>> > <security>
>>> > <user-name>keycloak_user</user-name>
>>> > <password><some-passphrase></password>
>>> > </security>
>>> > <validation>
>>> > <check-valid-connection-sql>select
>>> > 1</check-valid-connection-sql>
>>> > <validate-on-match>true</validate-on-match>
>>> >
>>> <background-validation>true</background-validation>
>>> >
>>> > <background-validation-millis>10000</background-validation-millis>
>>> > </validation>
>>> > </datasource>
>>> > <drivers>
>>> > <!-- driver declaration -->
>>> > <driver name="mariadb" module="org.mariadb">
>>> >
>>> > <xa-datasource-class>org.mariadb.jdbc.Driver</xa-datasource-class>
>>> > </driver>
>>> > <driver name="h2" module="com.h2database.h2">
>>> >
>>> > <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-datasource-class>
>>> > </driver>
>>> > </drivers>
>>> > </datasources>
>>> >
>>> >
>>> >
>>> > I'm using the mariadb-java-client-2.2.3 driver.
>>> >
>>> >
>>> >
>>> > <?xml version="1.0" ?>
>>> > <module xmlns="urn:jboss:module:1.3" name="org.mariadb">
>>> >
>>> > <resources>
>>> > <resource-root path="mariadb-java-client-2.2.3.jar"/>
>>> > </resources>
>>> >
>>> > <dependencies>
>>> > <module name="javax.api"/>
>>> > <module name="javax.transaction.api"/>
>>> > </dependencies>
>>> > </module>
>>> >
>>> >
>>> > Any assistance would be appreciated. I'll grab whatever information is
>>> > needed. Thank you in advance. :)
>>> > --
>>> > *Aaron Echols*
>>> > _______________________________________________
>>> > keycloak-user mailing list
>>> > keycloak-user at lists.jboss.org
>>> > https://lists.jboss.org/mailman/listinfo/keycloak-user
>>>
>>
More information about the keycloak-user
mailing list