[keycloak-user] Keycloak standalone-ha cluster jgroups without multicast
Christian Schneider
cschneiderpublic at gmail.com
Mon Jul 24 06:42:23 EDT 2017
That was the mistake. Thank you very much.
I used -Djboss.bind.address.private=xxx and it works fine now.
Best Regards,
Christian.
Meissa M'baye Sakho <msakho at redhat.com> schrieb am Mo., 24. Juli 2017 um
12:00 Uhr:
> Christian,
> There is a problem with IP that you have (not) defined for the private
> interface that is used by jgroup for intracluster communication.
> By default, JGroups only binds to the private network interface, which
> points to localhost in the default configuration (line 505 in your
> configuration file). For security reasons, *JGroups will not bind to the
> network interface defined by the -b argument specified during JBoss EAP
> startup*, as clustering traffic should not be exposed on a public network
> interface.
>
> You can define this value either by setting the
> property jboss.bind.address.private at startup or by hard coding it in the
> line 505 in you configuration file:
>
> 1. <interface name="private">
> 2. <inet-address value=
> "${jboss.bind.address.private:127.0.0.1}"/>
> 3. <inet-address value="your-private-ip"/>
> 4. </interface>
> 5.
>
>
> Doing so you should have an output like (instead of localhost in your
> case).
>
> *11:18:25,954 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-7) ISPN000079: Channel hibernate local address is app02.2.xxx.net
> <http://app02.2.xxx.net>, physical addresses are [your-private-ip:7600]*
>
> *11:18:25,954 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-4) ISPN000079: Channel ejb local address is app02.2.xxx.net
> <http://app02.2.xxx.net>, physical addresses are [your-private-ip:7600]*
>
> *11:18:25,954 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-2) ISPN000079: Channel web local address is app02.2.xxx.net
> <http://app02.2.xxx.net>, physical addresses are [your-private-ip:7600]*
>
> *11:18:25,955 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-8) ISPN000079: Channel server local address is app02.2.xxx.net
> <http://app02.2.xxx.net>, physical addresses are [your-private-ip:7600]*
>
> *11:18:25,955 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service
> thread 1-3) ISPN000079: Channel keycloak local address is app02.2.xxx.net
> <http://app02.2.xxx.net>, physical addresses are [1your-private-ip:7600]*
>
> *If your configuration jgroup configuration is ok and if you have defined
> the jboss.node.name <http://jboss.node.name> property for each of your
> cluster member, you should see in you log the output below that will
> confirm you that you members will form a cluster:*
>
> *11:18:45,074 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (thread-2,ee,tcpcluster2) ISPN000094: Received new cluster view for channel
> server: [tcpcluster2|1] (2) [tcpcluster2, tcpcluster1]*
>
> *11:18:45,076 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (thread-2,ee,tcpcluster2) ISPN000094: Received new cluster view for channel
> keycloak: [tcpcluster2|1] (2) [tcpcluster2, tcpcluster1]*
>
> *11:18:45,078 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (thread-2,ee,tcpcluster2) ISPN000094: Received new cluster view for channel
> web: [tcpcluster2|1] (2) [tcpcluster2, tcpcluster1]*
>
> *11:18:45,080 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (thread-2,ee,tcpcluster2) ISPN000094: Received new cluster view for channel
> hibernate: [tcpcluster2|1] (2) [tcpcluster2, tcpcluster1]*
>
> *11:18:45,080 INFO
> [org.infinispan.remoting.transport.jgroups.JGroupsTransport]
> (thread-2,ee,tcpcluster2) ISPN000094: Received new cluster view for channel
> ejb: [tcpcluster2|1] (2) [tcpcluster2, tcpcluster1]*
>
> *ISPN000310: Starting cluster-wide rebalance for cache work, topology
> CacheTopology{id=1, rebalanceId=1, currentCH=ReplicatedConsistentHash{ns =
> 60, owners = (1)[tcpcluster2: 60]}, pendingCH=ReplicatedConsistentHash{ns =
> 60, owners = (2)[tcpcluster2: 30, tcpcluster1: 30]}, unionCH=null,
> actualMembers=[tcpcluster2, tcpcluster1]}*
>
> In my case, I've defined the jboss.node.name respectively with
> tcpcluster1 and tcpcluster2.
>
>
> Meissa
>
>
>
> On Mon, Jul 24, 2017 at 9:26 AM, Marek Posolda <mposolda at redhat.com>
> wrote:
>
>> From the log, it seem that the servers don't see each other and hence
>> don't form cluster. In your configuration, you have
>> "10.62.168.51[7600],10.62.168.52[7600]", however in log later you have:
>>
>> physical addresses are [
>> 127.0.0.1:7600
>>
>> It seems there is the issue with the binding address? It should be
>> 10.62.168.51 instead of 127.0.0.1. Did you use the "-b" option when
>> starting the server?
>>
>> I think that JGroups/Infinispan/EAP documentation should have more
>> details as it's more related to that rather then to Keycloak.
>>
>> Marek
>>
>>
>> Dne 23.7.2017 v 12:51 Christian Schneider napsal(a):
>> > Hi,
>> > I'm trying to setup a keycloak cluster.
>> >
>> > We copy binaries and configuration through our build pipeline to the
>> > servers, so we decided to for the "standalone clustered mode".
>> >
>> > Since our operations don't enable ip multicast, we wan't to work with ip
>> > static ip addresses.
>> >
>> > The only source for this I found is:
>> > https://kb.novaordis.com/index.php/WildFly_Clustering_without_Multicast
>> > <
>> https://deref-gmx.net/mail/client/Wiot_0qYjM4/dereferrer/?redirectUrl=https%3A%2F%2Fkb.novaordis.com%2Findex.php%2FWildFly_Clustering_without_Multicast
>> >
>> >
>> > I configured both servers like this:
>> >
>> > <subsystem xmlns="urn:jboss:domain:jgroups:4.0">
>> > <channels default="ee">
>> > <channel name="ee" stack="tcp"/>
>> > </channels>
>> > <stacks default="tcp">
>> > <stack name="tcp">
>> > <transfer type="TCP" socket-binding="jgroups-tcp"/>
>> > <protocol type="TCPPING">
>> > <property
>> > name="initial_hosts">10.62.168.51[7600],10.62.168.52[7600]</property>
>> > <property
>> name="num_initial_members">2</property>
>> > <property name="port_range">0</property>
>> > <property name="timeout">2000</property>
>> > </protocol>
>> > <protocol type="MERGE3"/>
>> > <protocol type="FD_SOCK"
>> > socket-binding="jgroups-tcp-fd"/>
>> > <protocol type="FD"/>
>> > <protocol type="VERIFY_SUSPECT"/>
>> > <protocol type="pbcast.NAKACK2"/>
>> > <protocol type="UNICAST3"/>
>> > <protocol type="pbcast.STABLE"/>
>> > <protocol type="pbcast.GMS"/>
>> > <protocol type="MFC"/>
>> > <protocol type="FRAG2"/>
>> > </stack>
>> > </stacks>
>> > </subsystem>
>> >
>> > I can't figure out whether the two nodes paired each other. In the
>> logfile
>> > I only find this:
>> >
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000078:
>> > Starting JGroups channel hibernate
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000078:
>> > Starting JGroups channel web
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000078:
>> > Starting JGroups channel keycloak
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000078:
>> > Starting JGroups channel ejb
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000078:
>> > Starting JGroups channel server
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000094:
>> > Received new cluster view for channel keycloak: [app02.2.xxx.net|0]
>> (1) [
>> > app02.2.xxx.net]
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000094:
>> > Received new cluster view for channel hibernate: [app02.2.xxx.net|0]
>> (1) [
>> > app02.2.xxx.net]
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000094:
>> > Received new cluster view for channel ejb: [app02.2.xxx.net|0] (1) [
>> > app02.2.xxx.net]
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000094:
>> > Received new cluster view for channel server: [app02.2.xxx.net|0] (1) [
>> > app02.2.xxx.net]
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000094:
>> > Received new cluster view for channel web: [app02.2.xxx.net|0] (1) [
>> > app02.2.xxx.net]
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000079:
>> > Channel server local address is app02.2.xxx.net, physical addresses
>> are [
>> > 127.0.0.1:7600]
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000079:
>> > Channel ejb local address is app02.2.xxx.net, physical addresses are [
>> > 127.0.0.1:7600]
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000079:
>> > Channel web local address is app02.2.xxx.net, physical addresses are [
>> > 127.0.0.1:7600]
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000079:
>> > Channel keycloak local address is app02.2.xxx.net, physical addresses
>> are [
>> > 127.0.0.1:7600]
>> > INFO 2017-07-21 20:35:43 []
>> > org.infinispan.remoting.transport.jgroups.JGroupsTransport ISPN000079:
>> > Channel hibernate local address is app02.2.xxx.net, physical addresses
>> are [
>> > 127.0.0.1:7600
>> >
>> > I checked that the ports 7060 are open.
>> > When I access just a single node, the setup works fine. With two nodes I
>> > get {"error":"invalid_grant","error_description":"Session not active"}
>> on
>> > one of them.
>> > The goal is that all userSessions are synced between the cluster nodes,
>> so
>> > that the loadbalancer can roundrobbin between all nodes.
>> > Since we have around 20k users only, it would be fine to store the
>> users in
>> > our database. But keycloak doesn't provide this, right?
>> > Do you have any idea how to figure out whats wrong?
>> >
>> > Best Regards,
>> > Christian.
>> >
>> > P.S.: You can find the whole configuration here:
>> > https://pastebin.com/WC46pXGp
>> > <
>> https://deref-gmx.net/mail/client/UIuwfhWwMZ0/dereferrer/?redirectUrl=https%3A%2F%2Fpastebin.com%2FWC46pXGp
>> >
>> > _______________________________________________
>> > keycloak-user mailing list
>> > keycloak-user at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>>
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>
>
More information about the keycloak-user
mailing list