You're welcome, glad it helped :) Good luck with Keycloak!
Dmitry
On Wed, 2018-07-18 at 12:32 -0700, Aaron Echols wrote:
Ok, I fixed a variable in my
/etc/default/wildfly.conf
Forgot to change the hostname in there:
# Hostname:
WILDFLY_HOST=srv-iam-02
Once I fixed that, the server started syncing immediately. Thanks for
helping point me in the right direction. :)
--
Aaron Echols
On Wed, Jul 18, 2018 at 12:25 PM Aaron Echols <aechols(a)bfcsaz.com>
wrote:
> Hi Dmitry,
>
> I did as you suggested, but something seems amiss. When looking
> under:
>
> MBeans > org.wildfly.clustering.infinispan > CacheManager >
> "keycloak" > CacheManager > Attributes > clusterMembers
>
> shows the same hosts 2x: [srv-iam-01, srv-iam-01], the later should
> be 02. The other option you said to look it didn't seem to actually
> exist:
>
> MBeans -> org.wildfly.clustering.infinispan -> Cache ->
"keycloak"
> -> Cache
>
> I'm still confused and looking through the configs to see if I can
> figure out what is going on. Thanks :)
> --
> Aaron Echols
> Lead Administrator (IT)
> Benjamin Franklin Charter School | IT
> Email: aechols(a)bfcsaz.com
> Phone: (480) 677-8400
>
Website: http://www.bfcsaz.com
> Support Email: techsupport(a)bfcsaz.com
> Support Portal:
https://bfcs.freshservice.com/support/home
> Common Questions:
https://bfcs.freshservice.com/support/solutions
> Forgot your password:
https://accounts.bfcsaz.com
>
>
>
>
>
> *CONFIDENTIALITY NOTICE: This e-mail message, including any
> attachments, is for the sole use of the intended recipient(s) and
> may contain confidential and privileged information. Any
> unauthorized review, copy, use, disclosure, or distribution is
> prohibited. If you are not the intended recipient, please contact
> the sender by reply e-mail and destroy all copies of the original
> message.
>
>
> On Tue, Jul 17, 2018 at 4:01 PM Aaron Echols <aechols(a)bfcsaz.com>
> wrote:
> > Hi Dmitry,
> >
> > Thanks for the reply!
> >
> > I just finished upgrading to 4.1.0 and the issue persists...
> >
> > Let me try running the console and take a look there and see what
> > it shows. I'll post back shortly. Thanks for the help!
> > --
> > Aaron Echols
> >
> > On Tue, Jul 17, 2018 at 3:58 PM Dmitry Telegin <dt(a)acutus.pro>
> > wrote:
> > > Hi Aaron,
> > >
> > > This all sounds very weird. Off the top of my head:
> > > - try latest Keycloak (4.1.0), is the issue reproducible?
> > > - Infinispan exposes quite a lot of stuff via JMX. Run JMC or
> > > JConsole,
> > > connect to the Keycloak process, go to MBeans ->
> > > org.wildfly.clustering.infinispan -> Cache -> "keycloak"
->
> > > Cache. How
> > > many caches are there? (should be 15 as of KC 4.1.0) Are they
> > > all
> > > running? Are there any abnormalities? Entries under
> > > CacheManager might
> > > be useful, too.
> > >
> > > Cheers,
> > > Dmitry Telegin
> > > CTO, Acutus s.r.o.
> > > Keycloak Consulting and Training
> > >
> > > Pod lipami street 339/52, 130 00 Prague 3, Czech Republic
> > > +42 (022) 888-30-71
> > > E-mail: info(a)acutus.pro
> > >
> > > On Tue, 2018-07-17 at 13:28 -0700, Aaron Echols wrote:
> > > > Hello All,
> > > >
> > > > I've successfully setup a cluster with 2 nodes. Everything is
> > > working
> > > > great, except for one issue I can't figure out. I'm starting
> > > to pull my
> > > > hair out and wanted to see if anyone else has seen the issue
> > > and how to
> > > > correct it.
> > > >
> > > > I've setup a user federation using Active Directory (Server
> > > 2016) using
> > > > Keycloak 3.4.3. They are load balanced behind Netscaler
> > > 12.0.x. Infinispan
> > > > seems to be working correctly. It's backed by a MariaDB
> > > 10.1.x, 3 node
> > > > cluster. Things I've noted:
> > > >
> > > > - I can create a local user and it syncs instantly between
> > > the KC 3.4.3
> > > > nodes
> > > > - Password syncs work, all changes to attributes sync, etc
> > > > - I change settings for the user federation I created and
> > > they DON'T
> > > > sync, so creating a mapper, changing a sync setting, etc,
> > > they have to be
> > > > changed by hand manually on each node.
> > > > - Same with Role and realm-management. I can apply a
> > > permission to a
> > > > group or user and it doesn't sync.
> > > > - If I restart the wildfly server, the changes to
> > > propagate to the
> > > > opposite node everytime.
> > > >
> > > >
> > > >
> > > > I deleted a custom role in the realm-management client, and
> > > it deleted it
> > > > from the database. On the secondary node, I saw the file was
> > > still listed,
> > > > even with hard refreshes of the browser. I clicked to delete
> > > the custom
> > > > role and got the following in the server.log:
> > > >
> > > >
> > > >
> > > > ERROR [org.keycloak.services.error.KeycloakErrorHandler]
> > > (default task-26)
> > > > Uncaught server error: java.lang.IllegalStateException: Not
> > > found in
> > > > database
> > > > at
> > > >
> > > org.keycloak.models.cache.infinispan.RoleAdapter.isUpdated(Role
> > > Adapter.java:66)
> > > > at
> > > >
> > > org.keycloak.models.cache.infinispan.RoleAdapter.getId(RoleAdap
> > > ter.java:105)
> > > > at
> > > >
> > > org.keycloak.models.cache.infinispan.RealmCacheSession.removeRo
> > > le(RealmCacheSession.java:736)
> > > > at
> > > >
> > > org.keycloak.models.cache.infinispan.ClientAdapter.removeRole(C
> > > lientAdapter.java:587)
> > > > at
> > > >
> > > org.keycloak.services.resources.admin.RoleResource.deleteRole(R
> > > oleResource.java:53)
> > > > at
> > > >
> > > org.keycloak.services.resources.admin.RoleByIdResource.deleteRo
> > > le(RoleByIdResource.java:115)
> > > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > at
> > > >
> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccesso
> > > rImpl.java:62)
> > > > at
> > > >
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMetho
> > > dAccessorImpl.java:43)
> > > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > > at
> > > >
> > > org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjecto
> > > rImpl.java:140)
> > > > at
> > > >
> > > org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(Re
> > > sourceMethodInvoker.java:295)
> > > > at
> > > >
> > > org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMe
> > > thodInvoker.java:249)
> > > > at
> > > >
> > > org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetOb
> > > ject(ResourceLocatorInvoker.java:138)
> > > > at
> > > >
> > > org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceL
> > > ocatorInvoker.java:107)
> > > > at
> > > >
> > > org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetOb
> > > ject(ResourceLocatorInvoker.java:133)
> > > > at
> > > >
> > > org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceL
> > > ocatorInvoker.java:107)
> > > > at
> > > >
> > > org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetOb
> > > ject(ResourceLocatorInvoker.java:133)
> > > > at
> > > >
> > > org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceL
> > > ocatorInvoker.java:101)
> > > > at
> > > >
> > > org.jboss.resteasy.core.SynchronousDispatcher.invoke(Synchronou
> > > sDispatcher.java:406)
> > > > at
> > > >
> > > org.jboss.resteasy.core.SynchronousDispatcher.invoke(Synchronou
> > > sDispatcher.java:213)
> > > > at
> > > >
> > > org.jboss.resteasy.plugins.server.servlet.ServletContainerDispa
> > > tcher.service(ServletContainerDispatcher.java:228)
> > > > at
> > > >
> > > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher
> > > .service(HttpServletDispatcher.java:56)
> > > > at
> > > >
> > > org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher
> > > .service(HttpServletDispatcher.java:51)
> > > > at
> > > javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.ServletHandler.handleRequest(Servl
> > > etHandler.java:85)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFi
> > > lter(FilterHandler.java:129)
> > > > at
> > > >
> > > org.keycloak.services.filters.KeycloakSessionServletFilter.doFi
> > > lter(KeycloakSessionServletFilter.java:90)
> > > > at
> > > >
> > > io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.j
> > > ava:61)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFi
> > > lter(FilterHandler.java:131)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.FilterHandler.handleRequest(Filter
> > > Handler.java:84)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.security.ServletSecurityRoleHandle
> > > r.handleRequest(ServletSecurityRoleHandler.java:62)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.ServletDispatchingHandler.handleRe
> > > quest(ServletDispatchingHandler.java:36)
> > > > at
> > > >
> > > org.wildfly.extension.undertow.security.SecurityContextAssociat
> > > ionHandler.handleRequest(SecurityContextAssociationHandler.java
> > > :78)
> > > > at
> > > >
> > > io.undertow.server.handlers.PredicateHandler.handleRequest(Pred
> > > icateHandler.java:43)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.security.SSLInformationAssociation
> > > Handler.handleRequest(SSLInformationAssociationHandler.java:131
> > > )
> > > > at
> > > >
> > > io.undertow.servlet.handlers.security.ServletAuthenticationCall
> > > Handler.handleRequest(ServletAuthenticationCallHandler.java:57)
> > > > at
> > > >
> > > io.undertow.server.handlers.PredicateHandler.handleRequest(Pred
> > > icateHandler.java:43)
> > > > at
> > > >
> > > io.undertow.security.handlers.AbstractConfidentialityHandler.ha
> > > ndleRequest(AbstractConfidentialityHandler.java:46)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.security.ServletConfidentialityCon
> > > straintHandler.handleRequest(ServletConfidentialityConstraintHa
> > > ndler.java:64)
> > > > at
> > > >
> > > io.undertow.security.handlers.AuthenticationMechanismsHandler.h
> > > andleRequest(AuthenticationMechanismsHandler.java:60)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.security.CachedAuthenticatedSessio
> > > nHandler.handleRequest(CachedAuthenticatedSessionHandler.java:7
> > > 7)
> > > > at
> > > >
> > > io.undertow.security.handlers.NotificationReceiverHandler.handl
> > > eRequest(NotificationReceiverHandler.java:50)
> > > > at
> > > >
> > > io.undertow.security.handlers.AbstractSecurityContextAssociatio
> > > nHandler.handleRequest(AbstractSecurityContextAssociationHandle
> > > r.java:43)
> > > > at
> > > >
> > > io.undertow.server.handlers.PredicateHandler.handleRequest(Pred
> > > icateHandler.java:43)
> > > > at
> > > >
> > > org.wildfly.extension.undertow.security.jacc.JACCContextIdHandl
> > > er.handleRequest(JACCContextIdHandler.java:61)
> > > > at
> > > >
> > > io.undertow.server.handlers.PredicateHandler.handleRequest(Pred
> > > icateHandler.java:43)
> > > > at
> > > >
> > > org.wildfly.extension.undertow.deployment.GlobalRequestControll
> > > erHandler.handleRequest(GlobalRequestControllerHandler.java:68)
> > > > at
> > > >
> > > io.undertow.server.handlers.PredicateHandler.handleRequest(Pred
> > > icateHandler.java:43)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.ServletInitialHandler.handleFirstR
> > > equest(ServletInitialHandler.java:292)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.ServletInitialHandler.access$100(S
> > > ervletInitialHandler.java:81)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.ServletInitialHandler$2.call(Servl
> > > etInitialHandler.java:138)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.ServletInitialHandler$2.call(Servl
> > > etInitialHandler.java:135)
> > > > at
> > > >
> > > io.undertow.servlet.core.ServletRequestContextThreadSetupAction
> > > $1.call(ServletRequestContextThreadSetupAction.java:48)
> > > > at
> > > >
> > > io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(C
> > > ontextClassLoaderSetupAction.java:43)
> > > > at
> > > >
> > > org.wildfly.extension.undertow.security.SecurityContextThreadSe
> > > tupAction.lambda$create$0(SecurityContextThreadSetupAction.java
> > > :105)
> > > > at
> > > >
> > > org.wildfly.extension.undertow.deployment.UndertowDeploymentInf
> > > oService$UndertowThreadSetupAction.lambda$create$0(UndertowDepl
> > > oymentInfoService.java:1508)
> > > > at
> > > >
> > > org.wildfly.extension.undertow.deployment.UndertowDeploymentInf
> > > oService$UndertowThreadSetupAction.lambda$create$0(UndertowDepl
> > > oymentInfoService.java:1508)
> > > > at
> > > >
> > > org.wildfly.extension.undertow.deployment.UndertowDeploymentInf
> > > oService$UndertowThreadSetupAction.lambda$create$0(UndertowDepl
> > > oymentInfoService.java:1508)
> > > > at
> > > >
> > > org.wildfly.extension.undertow.deployment.UndertowDeploymentInf
> > > oService$UndertowThreadSetupAction.lambda$create$0(UndertowDepl
> > > oymentInfoService.java:1508)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequ
> > > est(ServletInitialHandler.java:272)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.ServletInitialHandler.access$000(S
> > > ervletInitialHandler.java:81)
> > > > at
> > > >
> > > io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequ
> > > est(ServletInitialHandler.java:104)
> > > > at
> > > >
> > > io.undertow.server.Connectors.executeRootHandler(Connectors.jav
> > > a:326)
> > > > at
> > > >
> > > io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.
> > > java:812)
> > > > at
> > > >
> > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExe
> > > cutor.java:1149)
> > > > at
> > > >
> > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolEx
> > > ecutor.java:624)
> > > > at java.lang.Thread.run(Thread.java:748)
> > > >
> > > >
> > > >
> > > > I'm not sure if there is an issue with Infinispan or a sql
> > > connection
> > > > issue. I've included my SQL connection string as well:
> > > >
> > > >
> > > >
> > > > <datasource jndi-
> > > name="java:jboss/datasources/KeycloakDS"
> > > > pool-name="KeycloakDS" enabled="true" use-java-
> > > context="true">
> > > > <connection-url>jdbc:mariadb://
> > > >
> > > 10.5.30.202:3306/keycloak?useUnicode=yes;characterEncoding=UTF-
> > > 8;sessionVariables=wait_timeout=180;autoRe
> > > > connect=true</connection-url>
> > > > <driver>mariadb</driver>
> > > > <pool>
> > > >
<max-pool-size>20</max-pool-size>
> > > > </pool>
> > > > <security>
> > > >
<user-name>keycloak_user</user-name>
> > > > <password><some-
> > > passphrase></password>
> > > > </security>
> > > > <validation>
> > > > <check-valid-connection-sql>select
> > > > 1</check-valid-connection-sql>
> > > >
<validate-on-match>true</validate-on-
> > > match>
> > > > <background-
> > > validation>true</background-validation>
> > > >
> > > > <background-validation-millis>10000</background-validation-
> > > millis>
> > > > </validation>
> > > > </datasource>
> > > > <drivers>
> > > > <!-- driver declaration -->
> > > > <driver name="mariadb"
> > > module="org.mariadb">
> > > >
> > > >
<xa-datasource-class>org.mariadb.jdbc.Driver</xa-datasource-
> > > class>
> > > > </driver>
> > > > <driver name="h2"
> > > module="com.h2database.h2">
> > > >
> > > > <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-
> > > datasource-class>
> > > > </driver>
> > > > </drivers>
> > > > </datasources>
> > > >
> > > >
> > > >
> > > > I'm using the mariadb-java-client-2.2.3 driver.
> > > >
> > > >
> > > >
> > > > <?xml version="1.0" ?>
> > > > <module xmlns="urn:jboss:module:1.3"
name="org.mariadb">
> > > >
> > > > <resources>
> > > > <resource-root
path="mariadb-java-client-2.2.3.jar"/>
> > > > </resources>
> > > >
> > > > <dependencies>
> > > > <module name="javax.api"/>
> > > > <module name="javax.transaction.api"/>
> > > > </dependencies>
> > > > </module>
> > > >
> > > >
> > > > Any assistance would be appreciated. I'll grab whatever
> > > information is
> > > > needed. Thank you in advance. :)
> > > > --
> > > > *Aaron Echols*
> > > > _______________________________________________
> > > > keycloak-user mailing list
> > > > keycloak-user(a)lists.jboss.org
> > > >
https://lists.jboss.org/mailman/listinfo/keycloak-user
> > >