[keycloak-user] Domain mode cluster, slave authentication?

[Max Allan]

Hi,
Has anyone used the latest WildFly 11 version of Keycloak in domain mode?

I could get it to work on a local instance with host-master and host-slave
config files. But using the same host-slave config on a different instance
it would fail to authenticate.
Error :

[Host Controller] 09:07:25,741 INFO  [org.jboss.remoting] (MSC service
thread 1-1) JBoss Remoting version 5.0.5.Final

[Host Controller] 09:07:25,874 INFO  [org.jboss.as.remoting] (MSC service
thread 1-2) WFLYRMT0001: Listening on 127.0.0.1:3456

[Host Controller] 09:07:26,167 WARN  [org.jboss.as.host.controller]
(Controller Boot Thread) WFLYHC0001: Could not connect to remote domain
controller remote://192.168.33.10:9999: java.lang.IllegalStateException:
WFLYHC0043: Unable to connect due to authentication failure.

[Host Controller] at org.jboss.as.host.controller.
RemoteDomainConnectionService.rethrowIrrecoverableConnectionFailures(
RemoteDomainConnectionService.java:674)

[Host Controller] at org.jboss.as.host.controller.
RemoteDomainConnectionService.register(RemoteDomainConnectionService.
java:293)

[Host Controller] at org.jboss.as.host.controller.
DomainModelControllerService.connectToDomainMaster(
DomainModelControllerService.java:938)

[Host Controller] at org.jboss.as.host.controller.
DomainModelControllerService.boot(DomainModelControllerService.java:692)

[Host Controller] at org.jboss.as.controller.AbstractControllerService$1.
run(AbstractControllerService.java:370)

[Host Controller] at java.lang.Thread.run(Thread.java:748)

[Host Controller] Caused by: javax.security.sasl.SaslException:
Authentication failed: all available authentication mechanisms failed:

[Host Controller]    DIGEST-MD5: javax.security.sasl.SaslException:
DIGEST-MD5: Server rejected authentication

[Host Controller] at org.jboss.remoting3.remote.
ClientConnectionOpenListener.allMechanismsFailed(
ClientConnectionOpenListener.java:109)

[Host Controller] at org.jboss.remoting3.remote.
ClientConnectionOpenListener$Capabilities.handleEvent(
ClientConnectionOpenListener.java:446)

[Host Controller] at org.jboss.remoting3.remote.
ClientConnectionOpenListener$Capabilities.handleEvent(
ClientConnectionOpenListener.java:242)

[Host Controller] at org.xnio.ChannelListeners.invokeChannelListener(
ChannelListeners.java:92)

[Host Controller] at org.xnio.conduits.ReadReadyHandler$
ChannelListenerHandler.readReady(ReadReadyHandler.java:66)

[Host Controller] at org.xnio.nio.NioSocketConduit.
handleReady(NioSocketConduit.java:89)

[Host Controller] at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)

[Host Controller] at ...asynchronous invocation...(Unknown Source)

[Host Controller] at org.jboss.remoting3.EndpointImpl.connect(
EndpointImpl.java:570)

[Host Controller] at org.jboss.remoting3.EndpointImpl.connect(
EndpointImpl.java:532)

[Host Controller] at org.jboss.remoting3.EndpointImpl.connect(
EndpointImpl.java:520)

[Host Controller] at org.jboss.as.protocol.ProtocolConnectionUtils.connect(
ProtocolConnectionUtils.java:204)

[Host Controller] at org.jboss.as.protocol.ProtocolConnectionUtils.
connectSync(ProtocolConnectionUtils.java:120)

[Host Controller] at org.jboss.as.host.controller.
RemoteDomainConnection.lambda$openConnection$0(RemoteDomainConnection.java:
223)

[Host Controller] at org.wildfly.common.context.
Contextual.runExceptionAction(Contextual.java:108)

[Host Controller] at org.wildfly.security.auth.client.AuthenticationContext.
run(AuthenticationContext.java:268)

[Host Controller] at org.jboss.as.host.controller.RemoteDomainConnection.
openConnection(RemoteDomainConnection.java:223)

[Host Controller] at org.jboss.as.host.controller.RemoteDomainConnection$
InitialConnectTask.connect(RemoteDomainConnection.java:592)

[Host Controller] at org.jboss.as.protocol.ProtocolConnectionManager.
connect(ProtocolConnectionManager.java:70)

[Host Controller] at org.jboss.as.host.controller.RemoteDomainConnection.
connect(RemoteDomainConnection.java:147)

[Host Controller] at org.jboss.as.host.controller.
RemoteDomainConnectionService.register(RemoteDomainConnectionService.
java:288)

[Host Controller] ... 4 more

[Host Controller] Suppressed: javax.security.sasl.SaslException:
DIGEST-MD5: Server rejected authentication

[Host Controller] at org.jboss.remoting3.remote.
ClientConnectionOpenListener$Authentication.handleEvent(
ClientConnectionOpenListener.java:736)

[Host Controller] at org.jboss.remoting3.remote.
ClientConnectionOpenListener$Authentication.handleEvent(
ClientConnectionOpenListener.java:578)

[Host Controller] at org.xnio.ChannelListeners.invokeChannelListener(
ChannelListeners.java:92)

[Host Controller] at org.xnio.conduits.ReadReadyHandler$
ChannelListenerHandler.readReady(ReadReadyHandler.java:66)

[Host Controller] at org.xnio.nio.NioSocketConduit.
handleReady(NioSocketConduit.java:89)

[Host Controller] at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)

[Host Controller]

[Host Controller] 09:07:26,169 WARN  [org.jboss.as.host.controller]
(Controller Boot Thread) WFLYHC0147: No domain controller discovery options
remain.

[Host Controller] 09:07:26,169 ERROR [org.jboss.as.host.controller]
(Controller Boot Thread) WFLYHC0002: Could not connect to master. Error
was: java.lang.IllegalStateException: WFLYHC0120: Tried all domain
controller discovery option(s) but unable to connect

[Host Controller] 09:07:26,170 FATAL [org.jboss.as.host.controller]
(Controller Boot Thread) WFLYHC0178: Aborting with exit code 99

After poking around a bit I found the slave makes a connection with this
bit of host-slave.xml :

   <domain-controller>
        <remote username="$local" security-realm="ManagementRealm">
            <discovery-options>
                <static-discovery name="primary"
protocol="${jboss.domain.master.protocol:remote}"
host="${jboss.domain.master.address:192.168.33.10}"
port="${jboss.domain.master.port:9999}"/>
            </discovery-options>
        </remote>
    </domain-controller>


I changed $local to admin and it connects fine. But if I understand
Wildfly, then you shouldn't need to specify a username at all, and the
remote server will interpret $local (or none) as the local default user,
which would be "admin".
I tried leaving out the username and that didn't work either.

Have I managed to configure my user wrong on the master somehow? I used
this command (and obviously have the correct secret in the host-slave.xml) :

bin/add-user.sh -u admin -p password -r ManagementRealm -ds -e


Or is this actually expected behaviour?