infinispan as a storage mechanism
by Bill Burke
If we had a built-in, clusterable storage mechanism for Keycloak using
Infinispan we would:
* Shorten build times drastically. 30 minutes and growing for me for
JPA builds. Liquibase + JPA startup takes 5-7 seconds on my box.
* Simpler startup. No need to start a DB.
* Reduce memory footprint? I think JPA is responsible for a lot of
classes loaded.
I've started some work on this in spare time. I'd say I'd be done in
like 2 months considering the other work I have in queue.
Looking at FineGrainAtomicMap as an implementation. Should make DB
migration simple and replication quicker.
--
Bill Burke
Red Hat
6 years, 2 months
Re: [keycloak-dev] [IAM] Cross-datacenter configuration issues
by William Burns
----- Original Message -----
> From: "Brian Atkisson" <batkisso(a)redhat.com>
> To: "Marek Posolda" <mposolda(a)redhat.com>
> Cc: "William Burns" <wburns(a)redhat.com>, "Pedro Zapata Fernandez" <pzapataf(a)redhat.com>, "iam-list"
> <iam-list(a)redhat.com>, "keycloak-dev" <keycloak-dev(a)lists.jboss.org>, "Hynek Mlnarik" <hmlnarik(a)redhat.com>,
> "Tristan Tarrant" <ttarrant(a)redhat.com>, "Boleslaw Dawidowicz" <bdawidow(a)redhat.com>, mmccomas(a)redhat.com
> Sent: Friday, March 2, 2018 6:52:19 PM
> Subject: Re: [IAM] [keycloak-dev] Cross-datacenter configuration issues
>
> +Mike
>
> > On Mar 2, 2018, at 12:20 PM, Marek Posolda <mposolda(a)redhat.com> wrote:
> >
> >> On 02/03/18 15:24, William Burns wrote:
> >>> (3) Use the approach, which works for me and which I put in JIRA. So
> >>> have 2 hotrod endpoints on JDG side (unsecured and secured). RemoteStore
> >>> access the unsecured endpoint. Just some sources, which deals with
> >>> __script cache, access the secured hotrod endpoint. This requires to
> >>> "patch" module keycloak-model-infinispan on RHSSO side and also require
> >>> some smaller changes in the configuration (some new configuration
> >>> properties are added on RHSSO side to allow configuration of hotrod
> >>> endpoint security, username ,password etc).
> >>>
> >>> (4) Bypass JDG security entirely with some picketbox "anonymous access"
> >>> JAAS login modules. I didn't manage to have it working and it completely
> >>> bypass security. On the other hand, it seems to be the only solution,
> >>> which won't require changes on Keycloak/RHSSO side. But we don't know
> >>> yet if it works...
> >> Actually there is another way to bypass security that Tristan brought up
> >> that we were was discussing. For JDG 7.2.CR1 we can relax access to
> >> protected caches, such as scriptcache, so that there is no global
> >> authorization check (removing a single if block). This way the cache
> >> just uses the normal authorization checks if it is even enabled for that
> >> cache. Unfortunately CR1 is not going to be available for another 3 weeks
> >> though. In the mean time Jared and I discussed just testing with
> >> community server version 8.2.8 for the standalone install until we can
> >> get JDG 7.2.CR1 installed there. He may even want to test with something
> >> a little newer.
> > Thanks for the update.
> >
> > If Jared is fine to temporarily test with Infinispan server 8.2.8, it's
> > cool. We will try to see if we can bypass security even for JDG 7.1 (Hynek
> > is looking into picketbox "anonymous" modules) and there is this patch
> > from me to secure access to "_script" cache, which works with 7.1, but
> > will require to patch Keycloak/RHSSO.
> >
> > I afraid we can officially update to JDG 7.2 at the moment, when 7.2.GA is
> > released. However it may be good to test early with 7.2, so we can find
> > earlier if some fixes are needed on JDG side and notify you before GA.
> > Hopefully we have time for trying that... :)
> >
> >
> > BTV. Will JDG 7.2 have better "pagination" support for RemoteCaches, so I
> > can do something like:
> >
> > remoteCache.keySet().stream().skip(100).limit(10).collect(Collectors.toList());
The remote client added support for the various keySet/entrySet/values collections in this version :)
The way this works is these implement bulk operations such as stream on top of the retrieveEntries method. This internally invokes [1] with a default batch size of [2]. You can override this default by setting the batch size of the remote cache [3][4] or by invoking the [1] method directly, passing your desired batch size and creating a stream from that.
If you want it to be the most efficient I would do [5] - this way it only pulls down 110 entries and only has the key as well (it uses the always registered converter that the keySet().iterator() uses). Unfortunately this points out there is no easy way to configure batch size for key iteration on the client (we need to fix that).
[1] https://github.com/infinispan/infinispan/blob/master/client/hotrod-client...
[2] https://github.com/infinispan/infinispan/commit/a5d8babfcca031227106c546a...
[3] https://github.com/infinispan/infinispan/blob/master/client/hotrod-client...
[4] https://github.com/infinispan/infinispan/commit/a5d8babfcca031227106c546a...
[5] https://gist.github.com/wburns/140e288f27ce7a4b04aec8ee65d9f819
By the way you should be able to do [5] even in ISPN 8.2.8 as well. The only thing that was added was the support of the standard collection methods on Map (ie. keySet, entrySet and values).
> >
> > to ensure that just 10 items from 100 to 110 are sent over the network? And
> > will it have Server tasks support? If yes, we will be hopefully able to
Hrmm, JDG 7.1 should have ServerTask, what was missing there?
> > get rid of remote scripting entirely :)
> >
> > But problem is also "client" (Keycloak) side as we rely on the infinispan
> > version provided by Wildfly/EAP. In other words, we are always few months
> > (or years?) behind all the cool features and fixes, which you are
> > currently developing in infinispan master :(
Yeah that is sad :(
> >
> > Marek
> >>
> >> Unfortunately that means that this version of keycloak would only work
> >> with JDG 7.2.0.CR1 or ISPN 9.0.3 [1] and newer. Either way I will be
> >> creating a PR for this on the JDG 7.2 branch today.
> >>
> >> [1] https://issues.jboss.org/browse/ISPN-7814
> >>
> >
>
6 years, 2 months
Client Adapters for Golang
by Bela Berde
Hi,
Is there someone who successfully connected Keycloak to a Golang
Frontend-Backend?
Many thanks.
Cheers
6 years, 2 months
Re: [keycloak-dev] Cross-datacenter configuration issues
by Marek Posolda
Thanks William for the help!
I've tried your approach (1) and I've updated just
infinispan-cachestore-remote to 9.1.6 and left the other dependencies
not updated. It failed for me with some NoClassDefFoundErrors. This is
not so surprising as the other infinispan modules and also
wildfly-clustering integration and parser modules were still using the
old 8.2.8 versions and there seem to be some API incompatibility though,
which break things. Here is stacktrace if you want to take a look:
https://pastebin.com/hzDSX2um
In the end, I've chosen a bit different solution, which works for me.
The RemoteStore of 8.2.8 doesn't support "security" but the RemoteCache
of 8.2.8 supports that. So I've changed some integration code on
Keycloak side to read RemoteCache configuration, which is retrieved from
the RemoteStore, and use it as a configuration template. Then I've
programatically added security into it and retrieve "secured"
RemoteCacheManager to access '__script_cache' and some parts of our
code, which need to execute remote scripts. See [1].
The RemoteStore itself is not secured. So on JDG side, I need to keep 2
hotrod endpoints (one unsecured accessed by RemoteStores and second
secured accessed by __script cache). I've added setup instructions to
the JIRA: [2] . Jared, I hope this can help you to have things working.
There maybe also another approach, which Hynek pointed, that we will
somehow bypass security on JDG side at all. Picketbox has some login
modules like org.jboss.security.auth.spi.AnonLoginModule and
org.jboss.security.auth.spi.IdentityLoginModule, which might allow to
configure JAAS realm in a way that "anonymous" user will be
automatically authenticated and role ___script_manager' granted to him.
In other words, this is completely bypass security. But possibly it may
allow to have things working without require changes on Keycloak code.
Another thing is, that I didn't manage to have it working, but I didn't
try to deeply and I am not too familiar with the Wildfly security +
picketbox stuff.
So to summary, The approaches I see is:
(1) Upgrade all infinispan modules in Keycloak to use 9.1.6 (or JDG
7.2.ER4). Will require to upgrade wildfly infinispan subsystem and bunch
of other dependencies (maybe whole EAP in the end). I guess it's not an
option regarding support, testing etc. On the other hand, it will allow
to configure RemoteStore security OOTB.
(2) Create our own subclass of Infinispan RemoteStore on Keycloak side,
which will allow to configure secure RemoteCache instances. Will require
changes on Keycloak sources, and also in the Keycloak servers
configuration, which will need to use new implementation of RemoteStore,
so the configuration of the RemoteStore will be completely changed.
(3) Use the approach, which works for me and which I put in JIRA. So
have 2 hotrod endpoints on JDG side (unsecured and secured). RemoteStore
access the unsecured endpoint. Just some sources, which deals with
__script cache, access the secured hotrod endpoint. This requires to
"patch" module keycloak-model-infinispan on RHSSO side and also require
some smaller changes in the configuration (some new configuration
properties are added on RHSSO side to allow configuration of hotrod
endpoint security, username ,password etc).
(4) Bypass JDG security entirely with some picketbox "anonymous access"
JAAS login modules. I didn't manage to have it working and it completely
bypass security. On the other hand, it seems to be the only solution,
which won't require changes on Keycloak/RHSSO side. But we don't know
yet if it works...
For now, my vote is to go with (3) and it's the only working. Any more
options?
[1]
https://github.com/mposolda/keycloak/blob/jdg-security/model/infinispan/s...
[2]
https://issues.jboss.org/browse/KEYCLOAK-6783?focusedCommentId=13540721&p...
Marek
On 01/03/18 15:42, William Burns wrote:
> So I was helping out Jared when found this issue.
>
> This is caused by a combination of issues due to varying versions (Infinispaan 8.2.8 and JDG 7.1)
>
> 1. JDG 7.1 only allows accessing internal caches remotely over loopback or when authorization is enabled. This came about in Infinispan 9.0 in [a]
> 2. Infinispan 8.2.6 does not allow for remote cache stores to use authentication. This was added in Infinispan 9.0.(2|3) and 9.1 in [b]
>
> Unfortunately with the combination of the 2 there is now way for an ISPN pre 9.0.2 from accessing a remote server 9.0 internal caches not on loopback, such as the script cache.
>
> There are 2 ways that I was able to come up with (thanks Tristan for the second)
>
> 1. Replace the infinispan-cachestore-remote.jar in the ISPN 8.2.6 install to have a newer version such as 9.1.6.Final at [c]. This should allow the 8.2.8 instance to use authorization on the remote store. I have not tested this, but everything should be contained in the jar (including parser).
> 2. Use JDG 7.2.ER4 or newer in the keycloak EAP instances as a module. This is the first version that allows for the user to deploy their own version of JDG as a module accessible over JNDI.
>
> - Will
>
> [a] https://issues.jboss.org/browse/ISPN-6457
> [b] https://issues.jboss.org/browse/ISPN-7868
> [c] https://mvnrepository.com/artifact/org.infinispan/infinispan-cachestore-r...
>
> ----- Original Message -----
>> From: "Hynek Mlnarik" <hmlnarik(a)redhat.com>
>> To: "Jared Blashka" <jblashka(a)redhat.com>
>> Cc: "Marek Posolda" <mposolda(a)redhat.com>, "Boleslaw Dawidowicz" <bdawidow(a)redhat.com>, "keycloak-dev"
>> <keycloak-dev(a)lists.jboss.org>, "William Burns" <wburns(a)redhat.com>, "iam-list" <iam-list(a)redhat.com>
>> Sent: Thursday, March 1, 2018 3:33:17 AM
>> Subject: Re: [keycloak-dev] Cross-datacenter configuration issues
>>
>> I can only agree that it seems to be a difference between Infinispan server
>> and JDG, since we did test it in Amazon where each instance of Keycloak and
>> Infinispan was installed on separate VM [1]. Whether this difference might
>> be indeed there should be confirmed by someone from JDG team. William,
>> could you please comment here?
>>
>> [1]
>> http://blog.keycloak.org/2018/01/keycloak-cross-data-center-setup-in-aws....
>>
>> On Thu, Mar 1, 2018 at 9:25 AM, Marek Posolda <mposolda(a)redhat.com> wrote:
>>
>>> I've just simulated the issue and created
>>> https://issues.jboss.org/browse/KEYCLOAK-6783 . I am looking at it.
>>>
>>> What works and what we tested is:
>>>
>>> * Setup with infinispan-server-8.2.8 on "local" network (infinispan
>>> server bind on loopback address like "localhost" . Different
>>> infinispan servers running on the same laptop, but on various port
>>> offsets)
>>>
>>> * Setup with JDG server 7.1.0 on "local" network (JDG server bound on
>>> loopback address like "localhost" . Different JDG servers running on
>>> the same laptop, but on various port offsets)
>>>
>>> * Setup with infinispan-server-8.2.8 on "real" network (testing with
>>> infinispan hosts bound to real host with IP addresses like 192.168.0.1
>>> )
>>>
>>> We didn't test the combination with JDG server bind on "real" addresses
>>> and this is the only one where the issue happens
>>>
>>> It seems JDG 7.1.0 has some additional security when compared with the
>>> community infinispan-server 8.2.8 .
>>>
>>> The easiest workaround for you might be to test with community
>>> infinispan-server 8.2.8 instead of JDG 7.1.0 . Server can be downloaded
>>> from this address:
>>> http://downloads.jboss.org/infinispan/8.2.8.Final/
>>> infinispan-server-8.2.8.Final-bin.zip
>>> .
>>>
>>> I hope to update you later today once I have some more info. Thanks for
>>> the report and all the details you mentioned.
>>>
>>> Marek
>>>
>>>
>>> On 28/02/18 21:36, Jared Blashka wrote:
>>>> Hey all,
>>>>
>>>> I'm working on testing out the cross-datacenter replication
>>>> configuration in our development environment and I'm running into some
>>>> issues.
>>>>
>>>> I stood up some JDG 7.1 instances and some RH-SSO 7.2 instances all
>>>> running on my localhost all with different port offsets, followed the
>>>> instructions[1], and everything seemed to work well enough.
>>>>
>>>> Once I got beyond that and tried running RH-SSO and JDG on separate
>>>> servers I started running into issues[2] during RH-SSO startup. Looks
>>>> like RH-SSO is unable to connect to the remote ___script_cache but
>>>> that cache isn't mentioned anywhere in the RH-SSO documentation. The
>>>> error message (and online searching) indicates that this cache only
>>>> allows remote connections if authorization is enabled. I didn't see
>>>> any mention of configuration related to authentication or security for
>>>> the remote caches in the documentation either.
>>>>
>>>> At this point we roped in a JDG expert (cc'ed here) and found some
>>>> additional Infinispan documentation[3] on how to add authentication to
>>>> the *remote* caches within the JDG configuration but nothing much in
>>>> the way of adding authentication to the client cache configuration
>>>> inside RH-SSO that didn't involve programmatic changes. After some
>>>> additional searching we found some info[4] detailing how to add
>>>> security configurations to a remote-cache configuration in Infinispan
>>>> *9.1* but EAP 7.1 is only running Infinispan *8.2* which doesn't have
>>>> these changes.
>>>>
>>>> How did you get this working?
>>>>
>>>> Jared Blashka - Identity & Access Management
>>>>
>>>>
>>>> [1]
>>>> https://access.redhat.com/documentation/en-us/red_hat_
>>> single_sign-on/7.2/pdf/server_installation_and_
>>> configuration_guide/Red_Hat_Single_Sign-On-7.2-Server_Installation_and_
>>> Configuration_Guide-en-US.pdf#__WKANCHOR_1e
>>>> [2] http://pastebin.test.redhat.com/559674
>>>> [3]
>>>> http://infinispan.org/docs/stable/server_guide/server_
>>> guide.html#general_concepts
>>>> [4]
>>>> https://docs.jboss.org/infinispan/9.1/configdocs/
>>> infinispan-cachestore-remote-config-9.1.html
>>>
>>>
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>
>>
>> --
>>
>> --Hynek
>>
6 years, 2 months
Pairwise clients and authorization services
by Martin Hardselius
Hi,
It seems like authorization services break when using them with a pairwise
enabled client. I've not investigated the full extent of this but long
story short, the sub from the token is used in token validation and in
org.keyclak.authorization.common.KeycloakIdentity for some comparisons.
Steps to reproduce:
1. Create pairwise a client with authorization enabled
3. Get access token (client_credentials)
3, Try post a new resource_set
I'm not sure what the best way to fix this is.
1. Re-write token validation and KeycloakIdentity to not rely on the sub in
the token,
2. Re-write the pairwise protocol mapper to ignore service accounts (feels
like putting make-up on a pig), or
3. "terminate" pairwise subs, replacing them with the internal sub, before
further processing.
Thoughts?
Regards,
Martin
6 years, 2 months
Non-UMA Use Cases
by Pedro Igor Silva
Hi,
There are situations where you don't want to use the UMA protocol to obtain
permissions from the server in form of an access token. Reasons can be: you
don't have any privacy requirements or your requirements don't require all
the UMA dance to obtain an access token.
In these situations, you may just want to send a request with the resources
and scopes you want to get access and get back the access token with these
permissions.
This is what we provide today with the Entitlement API, an alternative that
simplifies how clients can obtain permissions from the server.
However, with the introduction of UMA 2.0 support, we have now a specific
grant type [1] for obtaining permissions from the server using the token
endpoint. Just like any other OAuth2 grant type, the UMA grant type expects
a HTTP FORM request with some parameters.
The Entitlement API no longer exists but the same behavior can be achieved
with the new UMA grant type that was introduced. In other words, you can
use this grant type to ask for an access token with permissions without
sending a permission ticket but a list of resources/scopes (as parameters)
you want to get access.
The reason for reusing the grant type is that I would like to avoid having
two endpoints for obtaining permissions from the server. I think it makes
things simple.
Would like to know your opinion if moving the Entitlement API functionality
to this new grant type makes sense and if, maybe, we should have a specific
grant type (based on UMA grant type) that allows authorization requests
without a permission ticket (but a list of resources and scopes you want to
access). As side note, the code for UMA and non-UMA authorization is pretty
much the same, the main difference is on the format of the authorization
request/protocol.
[1] https://docs.kantarainitiative.org/uma/wg/oauth-uma-grant-2.0-09.html
Regards.
Pedro Igor
6 years, 2 months
Cross-datacenter configuration issues
by Jared Blashka
Hey all,
I'm working on testing out the cross-datacenter replication configuration
in our development environment and I'm running into some issues.
I stood up some JDG 7.1 instances and some RH-SSO 7.2 instances all running
on my localhost all with different port offsets, followed the
instructions[1], and everything seemed to work well enough.
Once I got beyond that and tried running RH-SSO and JDG on separate servers
I started running into issues[2] during RH-SSO startup. Looks like RH-SSO
is unable to connect to the remote ___script_cache but that cache isn't
mentioned anywhere in the RH-SSO documentation. The error message (and
online searching) indicates that this cache only allows remote connections
if authorization is enabled. I didn't see any mention of configuration
related to authentication or security for the remote caches in the
documentation either.
At this point we roped in a JDG expert (cc'ed here) and found some
additional Infinispan documentation[3] on how to add authentication to the
*remote* caches within the JDG configuration but nothing much in the way of
adding authentication to the client cache configuration inside RH-SSO that
didn't involve programmatic changes. After some additional searching we
found some info[4] detailing how to add security configurations to a
remote-cache configuration in Infinispan *9.1* but EAP 7.1 is only running
Infinispan *8.2* which doesn't have these changes.
How did you get this working?
Jared Blashka - Identity & Access Management
[1]
https://access.redhat.com/documentation/en-us/red_hat_single_sign-on/7.2/...
[2] http://pastebin.test.redhat.com/559674
[3]
http://infinispan.org/docs/stable/server_guide/server_guide.html#general_...
[4]
https://docs.jboss.org/infinispan/9.1/configdocs/infinispan-cachestore-re...
6 years, 2 months
