Enabling User Caching from Custom UserStorageProvider leads to MORE requests
by Rob Shepherd
Hi,
I’m having trouble getting UserModel caching to work properly. It seems like a bug.
I have a successful implementation of a UserStorageProvider using HTTP/REST calls to fetch user data. This has been working well in trials.
I have just enabled caching in the Realm > User Federation > Cache Settings > Cache Policy and selected a MAX_LIFESPAN time of 300’000ms (5mins)
I have found that for a test loop (1. login, 2. delete app session cookie, 3. authenticate by SSO cookie) this leads to MORE requests to the backend.
Below is the logging output which demonstrates this.
My question:
Could this be a bug?
or are there particular UserModel classes/interfaces that must be used to permit caching?
it seems like even for an object loaded in memory for a running thread/request it is not being found in the cache and lookup happens everytime the user object is required.
Here is the fetching and caching over that time…. with caching enabled first……
19:13:29,102 INFO [org.me.MyCustomUserStorageProvider] (default task-32) UserStorageProvider#getUserByEmail(): r...(a)gmail.com
19:13:34,637 INFO [org.me.MyCustomUserStorageProvider] (default task-43) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:34,706 INFO [org.me.MyCustomUserStorageProvider] (default task-38) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:34,746 INFO [org.me.MyCustomUserStorageProvider] (default task-38) Caching for delegate: MyUserAdapter:f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:34,754 INFO [org.me.MyCustomUserStorageProvider] (default task-38) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:34,783 INFO [org.me.MyCustomUserStorageProvider] (default task-38) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,454 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,496 INFO [org.me.MyCustomUserStorageProvider] (default task-45) Caching for delegate: MyUserAdapter:f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,501 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,540 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,570 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,605 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,631 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,661 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,689 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,714 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,751 INFO [org.me.MyCustomUserStorageProvider] (default task-45) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,832 INFO [org.me.MyCustomUserStorageProvider] (default task-50) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
19:13:37,882 INFO [org.me.MyCustomUserStorageProvider] (default task-50) Caching for delegate: MyUserAdapter:f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
That’s a total of 3 cache writes from 16 lookups for the same user in less than 1 minute, involving 3 requests to Keycloak in this session.
And now with the cache turned off and the cache flushed…..
18:57:45,343 INFO [org.me.MyCustomUserStorageProvider] (default task-58) UserStorageProvider#getUserByEmail(): r...(a)gmail.com
18:57:45,534 INFO [org.me.MyCustomUserStorageProvider] (default task-56) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
18:57:45,662 INFO [org.me.MyCustomUserStorageProvider] (default task-61) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
18:57:56,208 INFO [org.me.MyCustomUserStorageProvider] (default task-64) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
18:57:56,270 INFO [org.me.MyCustomUserStorageProvider] (default task-60) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
18:58:01,186 INFO [org.me.MyCustomUserStorageProvider] (default task-7) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
18:58:01,265 INFO [org.me.MyCustomUserStorageProvider] (default task-9) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
18:58:01,354 INFO [org.me.MyCustomUserStorageProvider] (default task-12) UserStorageProvider#getUserById(): f:d0c27624-05e4-4574-bc59-1911e9853917:5a04cf428c6bda1d05dbe0df
Thats a total of 8 lookups for the same user in less than 1 minute, involving 3 requests to Keycloak in this session.
6 years, 5 months
docker image and database - 3 starts?
by Rob Shepherd
Hi,
I’m running kc 3.3.0.FINAL using the docker image.
I’ve setup my database using the POSTGRES_* env vars.
I notice that everytime the server is started, there are actually 3 starts
I’m guessing the issue with WFLYCTL0212 is that the batch CLI change is attempting to set the jdbc driver parameters that already exists.
But why does it attempt it a second time before finally starting properly?
Presumably the way to workaround this is to just overwrite the parameters in standalone.xml directly, avoiding POSTGRES_* style env vars which will then ignore the change-database stuff?
here are the pertinent snippets…
1.
10:53:47,050 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
10:53:47,054 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Keycloak 3.3.0.Final (WildFly Core 3.0.8.Final) started in 7188ms - Started 67 of 79 services (23 services are lazy, passive or on-demand)
The batch failed with the following error: :
WFLYCTL0062: Composite operation failed and was rolled back. Steps that failed:
Step: step-9
Operation: /subsystem=datasources/jdbc-driver=postgresql:add(driver-name=postgresql, driver-module-name=org.postgresql.jdbc, driver-xa-datasource-class-name=org.postgresql.xa.PGXADataSource)
Failure: WFLYCTL0212: Duplicate resource [
("subsystem" => "datasources"),
("jdbc-driver" => "postgresql")
]
10:53:47,388 INFO [org.jboss.as] (MSC service thread 1-4) WFLYSRV0050: Keycloak 3.3.0.Final (WildFly Core 3.0.8.Final) stopped in 25ms
2.
10:54:00,061 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
10:54:00,067 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Keycloak 3.3.0.Final (WildFly Core 3.0.8.Final) started in 10751ms - Started 65 of 84 services (30 services are lazy, passive or on-demand)
The batch failed with the following error: :
WFLYCTL0062: Composite operation failed and was rolled back. Steps that failed:
Step: step-9
Operation: /subsystem=datasources/jdbc-driver=postgresql:add(driver-name=postgresql, driver-module-name=org.postgresql.jdbc, driver-xa-datasource-class-name=org.postgresql.xa.PGXADataSource)
Failure: WFLYCTL0212: Duplicate resource [
("subsystem" => "datasources"),
("jdbc-driver" => "postgresql")
]
10:54:00,720 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050: Keycloak 3.3.0.Final (WildFly Core 3.0.8.Final) stopped in 49ms
3.rd time lucky…
=========================================================================
JBoss Bootstrap Environment
JBOSS_HOME: /opt/jboss/keycloak
JAVA: /usr/lib/jvm/java/bin/java
JAVA_OPTS: -server -Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true -XX:MaxPermSize=256m
=========================================================================
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
10:54:03,396 INFO [org.jboss.modules] (main) JBoss Modules version 1.6.1.Final
10:54:04,483 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.7.SP1
10:54:04,989 INFO [org.jboss.as] (MSC service thread 1-1) WFLYSRV0049: Keycloak 3.3.0.Final (WildFly Core 3.0.8.Final) starting
6 years, 5 months
Complete export of a realm without shutdown
by Bart Smits
Hello,
I need to migrate a realm from a running Keycloak server.
I have been looking at the REST API, the command line and finally the startup options.
The startup option provide the best way to export a realm but it requires a restart.
Am I missing something or is this not yet supported?
Would it be possible to use bits and pieces of the command line or REST API to construct a complete export?
Kind regards,
Bart
6 years, 5 months
Re: [keycloak-user] Access Token getting truncated when apache HTTPD is in front
by Pharande Rahul
Hi,
Any updates/hints on this issue?
Thanks
Rahul
-----Original Message-----
From: Shaikh Asrafali Anwarali
Sent: Thursday, November 09, 2017 10:02 AM
To: stian(a)redhat.com; Pharande Rahul
Subject: RE: [keycloak-user] Access Token getting truncated when apache HTTPD is in front
Hi Stian,
Could you please share your views on the below issue, it's a blocker for us.
We have also posted this on keyclaok users forum, but we are still waiting for some kind of response.
Scroll downwards for issue detail.
Thanks in advance.
Regards,
Asraf Shaikh
-----Original Message-----
From: keycloak-user-bounces(a)lists.jboss.org [mailto:keycloak-user-bounces@lists.jboss.org] On Behalf Of Pharande Rahul
Sent: Wednesday, November 08, 2017 4:50 PM
To: keycloak-user(a)lists.jboss.org
Subject: [keycloak-user] Access Token getting truncated when apache HTTPD is in front
Hello Team,
I'm facing issue of "Access Token getting truncated when apache HTTPD is in front".
Though this issue is not directly associated/related to Keycloak but in combination with Apache HTTPD + Keycloak, I would like to take help from experts here :)
Below are more details on same.
Environnent :
o Server : Keycloak v3.x
o Proxy server : Apache HTTPD 2.4.x
o Client: Angular2 application using OIDC library.
Issue Description / Steps to reproduce:
* Create realm in Keycloak
* Create client for realm along with redirect url etc.
* Create ~70 role/permissions for client with longer names ~25 characters in permission name.
* Create user and assign all above permissions for newly created client.
* Access Angular2 application running in browser, and for protected resources Keycloak login page displayed where redirect_uri parameter is given/supplied.
* After entering valid user credentials, keycloak redirects to Application's redirect URL
* However error shown on browser console that, "failed at_hash".
o This is because incomplete/truncated token returned and OIDC client library in Angular application tries to validate token received.
Important point here:
* Defect mentioned only occurs when Apache is in front and used as proxy/load balancer server.
My analysis:
* As per my analysis, I see Keycloak returns access_token information in response header during redirect
* Apache has restriction of handling response header or cookies of size upto 8k
* Even after setting, various parameters in Apache HTTPD like - "LimitRequestFieldSize", "LimitRequestLine" we are still getting this error.
Please let me know if anyone already experienced such issue OR has any alternative on using/configuring Keycloak to redirect using part response..
Thanks and Regards.
Rahul Pharande
_______________________________________________
keycloak-user mailing list
keycloak-user(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-user
6 years, 5 months
Bug squashing time
by Stian Thorgersen
We're not accepting any contributions for new features until we start on
Keycloak 4.x. Most likely that'll be mid-December. Any PRs for new features
until then will not be reviewed or merged, but feel free to send if you
want.
We do however have a fair amount of outstanding bugs and would love help
from the community to get as many as possible resolved in the next month!
If you'd like to help take a look at
https://issues.jboss.org/projects/KEYCLOAK/versions/12333692 look for
issues that are not assigned to someone. Remember we want tests as well.
6 years, 5 months
Re: [keycloak-user] Authz with nodejs
by Corentin Dupont
OK problem solved: I forgot to check "Full Scope Allowed" in the client...
On Tue, Nov 7, 2017 at 10:03 PM, Corentin Dupont <corentin.dupont(a)gmail.com>
wrote:
> Hi guys,
> I created a REST API that I would like to protect with keycloak.
> However, I don't find any example/tutorial on Internet that suits.
>
> At the moment I use keycloak-nodejs-connect:
> https://github.com/keycloak/keycloak-nodejs-connect/blob/mas
> ter/example/index.js
>
> This is the basic example given:
>
> var Keycloak = require('keycloak-connect');
> var express = require('express');
> var session = require('express-session');
>
> var app = express();
> var server = app.listen(3000, function () {});
> var memoryStore = new session.MemoryStore();
>
> app.use(session({
> secret: 'mySecret',
> resave: false,
> saveUninitialized: true,
> store: memoryStore,
> }));
>
> var keycloak = new Keycloak({
> store: memoryStore
> });
>
> app.use(keycloak.middleware({
> logout: '/logout',
> admin: '/'
> }));
>
> app.get('/login', keycloak.protect(), function (req, res) {
> res.render('index', {
> result: JSON.stringify(JSON.parse(req.session['keycloak-token']),
> null, 4),
> event: '1. Authentication\n2. Login'
> });
> });
>
> But that doesn't corresponds to my need: in a REST API I have no login or
> logout and no memory.
> I think the user should always make requests with a bearer token. Based on
> that token I can identify the user and get his roles.
> Then I could use keycloak.protect('realm:myendpoint') to protect each of
> my endpoints. If the user have got that role, he is authorized.
> Did I understood correctly the flow?
> Is there some example or REST API with authz, using nodeJS?
>
> Thanks a lot!!!
> Corentin
>
6 years, 5 months
Is bearer token auth supported in commonly used REST client libraries?
by Vlastimil Elias
Hi,
is bearer token auth method supported in commonly used REST client
libraries (eg. RESTEasy and HttpClient in java)? I tried to google
around this, but no success. I was not able to find anything useful
(except stuff implemented inside of Keycloak Admin Client).
You know, http basic auth and ssl client cert auth is commonly available
and (relatively) easy to use, so it is widely used. I believe good, easy
to use, support of bearer token auth in client libraries is crucial to
drive adoption of this technology, including OpenID Connect protocol,
which then can drive use of Keycloak itself.
By "good, easy to use, support" I mean things like automatic obtaining
of access token based on the config (using OIDC "Client Credentials
Grant" and "Direct Access Grants"), its automatic renewal, and automatic
use of this token in REST calls performed by the client library, without
any complicated coding.
Any references to existing libraries? Any plans in Keycloak project itself?
Vlastimil
--
Vlastimil Elias
Principal Software Engineer, Middleware Engineering Services
Red Hat
6 years, 5 months
Can we map each realm into different URL?
by Min Han Lee
Hello Guys,
As title, does keycloak has out of the box functionality to do so? or do we
need to use a reverse proxy to execute this?
Any help is much appreciated
Kind Regards
6 years, 5 months
Federated and Dynamic Users/Attributes
by Andreas Tell
Hi!
In an upcoming system we aim to use Keycloak as a "OIDC/OAuth security
proxy/broker".
All information basically resides in other systems (federated);
* An external IdP provides ID federation via SAML v2
* Permissions are fetched dynamically each time the user authenticates from
an external system via a web service call. KC is not the system of record
for this information.
After the user is authenticated, the client (web app) retrieves the full
set of permissions info via the /userinfo endpoint by providing an Access
Token (resource owner credentials grant).
My first question is; is this approach at all advisable? Can it be done
using KC?
I got a clue from this ; https://stackoverflow.com/
questions/44014260/how-to-programmatically-assign-particular-roles-at-user-
registration-in-keycloak
If so I assume we'd have to extend KC using one of the SPIs.
The documentation on the SPIs don't give me much confidence on where to
best put such extensions.
Where would I put a web service call?
How can I dynamically assign roles and/or attributes to a provisioned user?
Should I use the Authentication SPI, User Federation SPI, User Storage SPI
or possibly piggyback on a callback event of Event Listener SPI ?
Best Regards
Andreas Tell
6 years, 5 months
identity broker role mapping bug?
by Simon Payne
Hi, i think i may have found a bug in the identity provider mapping of
claims to roles.
it appears that if i have an identity provider with claims in the token,
which i want to map to a role in the identity broker, then it only does
this once during the first time login. if i remove the claim from the
identity provider token, then this successfully removes it from the broker
- but never remaps if i then add it again.
the scenario i am trying to create here is that the identity provider is
responsible for authentication where active directory groups appears as
claim in the token. the broker then map this claim to the role providing
the authorization.
this behaviour appears to be the same whether i map a broker role to a
custom claim or a realm role in the provider token.
hope this makes sense, thanks
Simon.
6 years, 5 months
