[keycloak-dev] [IAM] Cross-datacenter configuration issues

[Marek Posolda]

On 03/03/18 23:40, William Burns wrote:
>
> ----- Original Message -----
>> From: "Brian Atkisson" <batkisso at redhat.com>
>> To: "Marek Posolda" <mposolda at redhat.com>
>> Cc: "William Burns" <wburns at redhat.com>, "Pedro Zapata Fernandez" <pzapataf at redhat.com>, "iam-list"
>> <iam-list at redhat.com>, "keycloak-dev" <keycloak-dev at lists.jboss.org>, "Hynek Mlnarik" <hmlnarik at redhat.com>,
>> "Tristan Tarrant" <ttarrant at redhat.com>, "Boleslaw Dawidowicz" <bdawidow at redhat.com>, mmccomas at redhat.com
>> Sent: Friday, March 2, 2018 6:52:19 PM
>> Subject: Re: [IAM] [keycloak-dev] Cross-datacenter configuration issues
>>
>> +Mike
>>
>>> On Mar 2, 2018, at 12:20 PM, Marek Posolda <mposolda at redhat.com> wrote:
>>>
>>>> On 02/03/18 15:24, William Burns wrote:
>>>>> (3) Use the approach, which works for me and which I put in JIRA. So
>>>>> have 2 hotrod endpoints on JDG side (unsecured and secured). RemoteStore
>>>>> access the unsecured endpoint. Just some sources, which deals with
>>>>> __script cache, access the secured hotrod endpoint. This requires to
>>>>> "patch" module keycloak-model-infinispan on RHSSO side and also require
>>>>> some smaller changes in the configuration (some new configuration
>>>>> properties are added on RHSSO side to allow configuration of hotrod
>>>>> endpoint security, username ,password etc).
>>>>>
>>>>> (4) Bypass JDG security entirely with some picketbox "anonymous access"
>>>>> JAAS login modules. I didn't manage to have it working and it completely
>>>>> bypass security. On the other hand, it seems to be the only solution,
>>>>> which won't require changes on Keycloak/RHSSO side. But we don't know
>>>>> yet if it works...
>>>> Actually there is another way to bypass security that Tristan brought up
>>>> that we were was discussing. For JDG 7.2.CR1 we can relax access to
>>>> protected caches, such as scriptcache, so that there is no global
>>>> authorization check (removing a single if block).  This way the cache
>>>> just uses the normal authorization checks if it is even enabled for that
>>>> cache. Unfortunately CR1 is not going to be available for another 3 weeks
>>>> though. In the mean time Jared and I discussed just testing with
>>>> community server version 8.2.8 for the standalone install until we can
>>>> get JDG 7.2.CR1 installed there. He may even want to test with something
>>>> a little newer.
>>> Thanks for the update.
>>>
>>> If Jared is fine to temporarily test with Infinispan server 8.2.8, it's
>>> cool. We will try to see if we can bypass security even for JDG 7.1 (Hynek
>>> is looking into picketbox "anonymous" modules) and there is this patch
>>> from me to secure access to "_script" cache, which works with 7.1, but
>>> will require to patch Keycloak/RHSSO.
>>>
>>> I afraid we can officially update to JDG 7.2 at the moment, when 7.2.GA is
>>> released. However it may be good to test early with 7.2, so we can find
>>> earlier if some fixes are needed on JDG side and notify you before GA.
>>> Hopefully we have time for trying that... :)
>>>
>>>
>>> BTV. Will JDG 7.2 have better "pagination" support for RemoteCaches, so I
>>> can do something like:
>>>
>>> remoteCache.keySet().stream().skip(100).limit(10).collect(Collectors.toList());
> The remote client added support for the various keySet/entrySet/values collections in this version :)
>
> The way this works is these implement bulk operations such as stream on top of the retrieveEntries method. This internally invokes [1] with a default batch size of [2]. You can override this default by setting the batch size of the remote cache [3][4] or by invoking the [1] method directly, passing your desired batch size and creating a stream from that.
>
> If you want it to be the most efficient I would do [5] - this way it only pulls down 110 entries and only has the key as well (it uses the always registered converter that the keySet().iterator() uses). Unfortunately this points out there is no easy way to configure batch size for key iteration on the client (we need to fix that).
>
> [1] https://github.com/infinispan/infinispan/blob/master/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/RemoteCache.java#L229
> [2] https://github.com/infinispan/infinispan/commit/a5d8babfcca031227106c546a3388fee2da4ee6e#diff-76807cffef27c52bdd3fafd782945895R75
> [3] https://github.com/infinispan/infinispan/blob/master/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/configuration/ConfigurationBuilder.java#L293
> [4] https://github.com/infinispan/infinispan/commit/a5d8babfcca031227106c546a3388fee2da4ee6e#diff-76807cffef27c52bdd3fafd782945895R65
> [5] https://gist.github.com/wburns/140e288f27ce7a4b04aec8ee65d9f819
>
> By the way you should be able to do [5] even in ISPN 8.2.8 as well. The only thing that was added was the support of the standard collection methods on Map (ie. keySet, entrySet and values).
Thanks for the pointers
>
>>> to ensure that just 10 items from 100 to 110 are sent over the network? And
>>> will it have Server tasks support? If yes, we will be hopefully able to
> Hrmm, JDG 7.1 should have ServerTask, what was missing there?
ah, indeed. I somehow missed that for some reason... Just checked that 
it requires a module with server task to be deployed on JDG side, which 
I am not sure works for us. The JS code has an advantage, that it can be 
deployed "on the fly" without additional need for packaging the JAR etc. 
But maybe it's way to go for us anyway as we may need some our classes 
to be available on JDG side for better filtering etc. I need to 
doublecheck that...

Marek
>
>>> get rid of remote scripting entirely :)
>>>
>>> But problem is also "client" (Keycloak) side as we rely on the infinispan
>>> version provided by Wildfly/EAP. In other words, we are always few months
>>> (or years?) behind all the cool features and fixes, which you are
>>> currently developing in infinispan master :(
> Yeah that is sad :(
>
>>> Marek
>>>> Unfortunately that means that this version of keycloak would only work
>>>> with JDG 7.2.0.CR1 or ISPN 9.0.3 [1] and newer. Either way I will be
>>>> creating a PR for this on the JDG 7.2 branch today.
>>>>
>>>> [1] https://issues.jboss.org/browse/ISPN-7814
>>>>