From sthorger at redhat.com  Mon Sep  3 06:16:32 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 3 Sep 2018 12:16:32 +0200
Subject: [keycloak-dev] SAML client integration HTTP 405 error
In-Reply-To: <DB5PR02MB16857315E8D1A6D8B0C5F1A49D0A0@DB5PR02MB1685.eurprd02.prod.outlook.com>
References: <DB5PR02MB16857315E8D1A6D8B0C5F1A49D0A0@DB5PR02MB1685.eurprd02.prod.outlook.com>
Message-ID: <CAJgngAcVKntTfC2VKUWkos4OEsjrEUNSqVgx_B4j9khOM3_tHw@mail.gmail.com>

Please use the user mailing list for general questions and help

On Tue, 28 Aug 2018 at 15:02, Adrien DESBIAUX <adesbiaux at vente-privee.com>
wrote:

> Hi there,
>
>
> We are running Keycloak 3.4, and we noticed that for some SAML clients, we
> get a HTTP 405 error.
>
>
> This happen for example with client having a local login and a button to
> login with SSO. For example Appdynamics does that (to name only that one).
> They perform a POST request to the Keycloak SAML endpoint of the configured
> client, leading to a HTTP 405 error.
>
>
> The fact that they perform a POST to load the Keycloak login page is
> discussable, but how to counter this behaviour on Keycloak client's
> configuration side?
>
>
> Thank you in advance for your guidance.
>
>
> Cheers,
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From sthorger at redhat.com  Mon Sep  3 06:21:20 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 3 Sep 2018 12:21:20 +0200
Subject: [keycloak-dev] SCIM v2 support
In-Reply-To: <93f62b8abc994e88835f314118fb9f4d@BOSKGEXC01.boskg.local>
References: <93f62b8abc994e88835f314118fb9f4d@BOSKGEXC01.boskg.local>
Message-ID: <CAJgngAd8-V++LvJijHR4a7nOk-KyqOg6qCA3HaG=uR-fYC0q9A@mail.gmail.com>

Adding additional attributes to user entity is probably not the way to go.
Rather, it would be better for backwards compatibility to simply use
generic key/value attributes which the user entity already has.

Implementing the SCIM endpoints is probably pretty straightforward. Most of
the work will probably be down to testing and documentation.

On Wed, 29 Aug 2018 at 13:28, L?sch, Sebastian <
Sebastian.Loesch at governikus.de> wrote:

> Hello,
>
>
>
> in a customer project we use keycloak and need a SCIM (System for
> Cross-domain Identity Management) API.
>
> Currently we write a wrapper API and a custom endpoint providing the SCIM
> functionality. We wrote a extension of the UserEntity, UserModel and an
> extension of the JpaUserProvider.
>
> This strategy seems not ideal and the nicest way is to add this extensions
> to Keycloak. This is already suggested in
> https://issues.jboss.org/browse/KEYCLOAK-2537
>
> Is anybody out there who can guide me, what coding would be necessary to
> contribute the SCIM functionality?
>
>
>
> Currently I think we have to:
>
> -          extend the UserEntity with all SCIM attributes. This will result
> in additional tables/entities for complex attributes e.g. Address, Name,
> Email
>
> -          extend the UserModel to povide the additional attributes
>
> -          implement the new SCIM endpoint /Users
>
> -          make the additional attributes available via Admin REST API
> /users
>
> -          extend views to be able to edit SCIM user attributes using the
> web ui
>
> -          ?
>
> -          All the above again for the Groups endpoint?
>
>
>
> This also seem to be major changes. To big for one Pull Request. How do you
> like to handle this?
>
>
>
> Best regards,
>
> Sebastian
>
>
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

From sthorger at redhat.com  Mon Sep  3 06:22:20 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 3 Sep 2018 12:22:20 +0200
Subject: [keycloak-dev] Implementing user self service via REST APIs
In-Reply-To: <CWXP265MB09527EE7F565356FE5F5CCF987080@CWXP265MB0952.GBRP265.PROD.OUTLOOK.COM>
References: <CWXP265MB09527EE7F565356FE5F5CCF987080@CWXP265MB0952.GBRP265.PROD.OUTLOOK.COM>
Message-ID: <CAJgngAfgTcK=0SAkUGnHqP6cYdn-h4R+zMwAAoJXFYzSpBvbFA@mail.gmail.com>

We are working on account management REST API. Some of it's already
available in master and the remaining functionality will come in the next
few releases.

On Thu, 30 Aug 2018 at 17:47, William Jones <william_jones20 at outlook.com>
wrote:

> Hi
>
> As part of my Keycloak implementation, I would like to offer user self
> service, e.g. change password.
>
> I do not wish to theme the built-in user account dashboard, but instead
> build the functionality directly into our website.
>
> We are already using the Admin API for user administration, but as I
> understand it, it would not be appropriate to use this for actions which
> are actually being carried out by the end user. The auditing would be
> incorrect, and certain functionality is unavailable anyway - for example,
> whilst we could set a new user password via the Admin API, we would have no
> way of verifying that the existing user password is correct (we want the
> user to provide existing and new, as per the user account dashboard).
>
> As such, is the correct approach to this for us to extend KeyCloak with a
> set of custom REST endpoints to be called by an end user rather than an
> admin? They will be authenticated at this point so we will be able to pass
> down their access token for the authentication.
>
> If so, I assume I should be following the instructions under "Add custom
> REST endpoints" detailed at the following URL?
>
> https://www.keycloak.org/docs/3.0/server_development/topics/extensions.html
> Extending Server | Keycloak Documentation<
> https://www.keycloak.org/docs/3.0/server_development/topics/extensions.html
> >
> This is a very powerful extension, which allows you to deploy your own
> REST endpoints to the Keycloak server. It enables all kinds of extensions,
> for example the possibility to trigger functionality on the Keycloak
> server, which is not available through the default set of built-in Keycloak
> REST endpoints.
> www.keycloak.org
>
>
> Thanks
>
> William
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From in.anand19 at gmail.com  Mon Sep  3 08:06:38 2018
From: in.anand19 at gmail.com (Mahendra Anand)
Date: Mon, 3 Sep 2018 17:36:38 +0530
Subject: [keycloak-dev] 403, Forbidden Issue with POST, PUT, DELETE
Message-ID: <CAK8h7=QnHRgnFVcfE01ZCk6os4r1czNyg7gFMygB=ciGax+Z6g@mail.gmail.com>

Hi Team,

I am trying to setup a standalone keycloak server and able to do it.
With the help of keycloak I am trying to secure REST endpoints which I am
exposing in my spring boot application.

I have all required steps to configure keycloak with spring boot
application link -
- creating new realm.
- Creating new client in that realm.
- new admin and user roles
- creating users with admin and user roles.

I am able to get access token with the help of admin user like below -

curl -d
"grant_type=password&client_id=product-app&username=admin&password=admin"
http://localhost:8181/auth/realms/springboot/protocol/openid-connect/token

And with the help of retrieved token i have able to hit GET end points of
my application.

But when I do POST, PUT, DELETE requests with token i get --
{
    "timestamp": "2018-09-03T11:27:16.266+0000",
    "status": 403,
    "error": "Forbidden",
    "message": "Forbidden",
    "path": "/ds/api/v1/template/create"
}
It might be a scope issue on the user I am creating, but I am not getting
any pointer to give correct scope to user.

Kindly suggest any pointer or help will be very much appreciated. Thanks!

-- 
Regards
Mahendra Anand
Mobile - +91 9711429614
Skype - mahendra.anand

From adesbiaux at vente-privee.com  Mon Sep  3 08:14:47 2018
From: adesbiaux at vente-privee.com (Adrien DESBIAUX)
Date: Mon, 3 Sep 2018 12:14:47 +0000
Subject: [keycloak-dev] SAML client integration HTTP 405 error
In-Reply-To: <CAJgngAcVKntTfC2VKUWkos4OEsjrEUNSqVgx_B4j9khOM3_tHw@mail.gmail.com>
References: <DB5PR02MB16857315E8D1A6D8B0C5F1A49D0A0@DB5PR02MB1685.eurprd02.prod.outlook.com>,
	<CAJgngAcVKntTfC2VKUWkos4OEsjrEUNSqVgx_B4j9khOM3_tHw@mail.gmail.com>
Message-ID: <DB5PR02MB16852085A44408CC6D8536B89D0C0@DB5PR02MB1685.eurprd02.prod.outlook.com>

Hi Stian,


I would think that my question directly referred to an implementation detail in Keycloak itself.


But thank you for your time.






________________________________
From: Stian Thorgersen <sthorger at redhat.com>
Sent: Monday, September 3, 2018 12:16:32 PM
To: Adrien DESBIAUX
Cc: keycloak-dev
Subject: Re: [keycloak-dev] SAML client integration HTTP 405 error

Please use the user mailing list for general questions and help

On Tue, 28 Aug 2018 at 15:02, Adrien DESBIAUX <adesbiaux at vente-privee.com<mailto:adesbiaux at vente-privee.com>> wrote:
Hi there,


We are running Keycloak 3.4, and we noticed that for some SAML clients, we get a HTTP 405 error.


This happen for example with client having a local login and a button to login with SSO. For example Appdynamics does that (to name only that one). They perform a POST request to the Keycloak SAML endpoint of the configured client, leading to a HTTP 405 error.


The fact that they perform a POST to load the Keycloak login page is discussable, but how to counter this behaviour on Keycloak client's configuration side?


Thank you in advance for your guidance.


Cheers,
_______________________________________________
keycloak-dev mailing list
keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-dev<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.jboss.org_mailman_listinfo_keycloak-2Ddev&d=DwMFaQ&c=8G9fL-FODW3I-fESjFvsaw&r=BhQIS0sALperhBRJRNzxM3Ax9g4k8rNIuzNmM35TMgE&m=72xRw4Glpv6dKcdXHkNdMWSdUO2SnV-al0GTcZGzDiE&s=_Q5d2-MpbXWlpKgBC0-nBHGbv6HpUu6ucLAhQjfFK-o&e=>

From hmlnarik at redhat.com  Mon Sep  3 08:33:05 2018
From: hmlnarik at redhat.com (Hynek Mlnarik)
Date: Mon, 3 Sep 2018 14:33:05 +0200
Subject: [keycloak-dev] SAML client integration HTTP 405 error
In-Reply-To: <DB5PR02MB16852085A44408CC6D8536B89D0C0@DB5PR02MB1685.eurprd02.prod.outlook.com>
References: <DB5PR02MB16857315E8D1A6D8B0C5F1A49D0A0@DB5PR02MB1685.eurprd02.prod.outlook.com>
	<CAJgngAcVKntTfC2VKUWkos4OEsjrEUNSqVgx_B4j9khOM3_tHw@mail.gmail.com>
	<DB5PR02MB16852085A44408CC6D8536B89D0C0@DB5PR02MB1685.eurprd02.prod.outlook.com>
Message-ID: <CAMvXD=GULVFXQFahOLMVtNKesqZpYZXgJ+-xWrS5YBhvsy=FRg@mail.gmail.com>

Please send this use-case question to keycloak-user with relevant details
of configurations and error logs.

On Mon, Sep 3, 2018 at 2:25 PM Adrien DESBIAUX <adesbiaux at vente-privee.com>
wrote:

> Hi Stian,
>
>
> I would think that my question directly referred to an implementation
> detail in Keycloak itself.
>
>
> But thank you for your time.
>
>
>
>
>
>
> ________________________________
> From: Stian Thorgersen <sthorger at redhat.com>
> Sent: Monday, September 3, 2018 12:16:32 PM
> To: Adrien DESBIAUX
> Cc: keycloak-dev
> Subject: Re: [keycloak-dev] SAML client integration HTTP 405 error
>
> Please use the user mailing list for general questions and help
>
> On Tue, 28 Aug 2018 at 15:02, Adrien DESBIAUX <adesbiaux at vente-privee.com
> <mailto:adesbiaux at vente-privee.com>> wrote:
> Hi there,
>
>
> We are running Keycloak 3.4, and we noticed that for some SAML clients, we
> get a HTTP 405 error.
>
>
> This happen for example with client having a local login and a button to
> login with SSO. For example Appdynamics does that (to name only that one).
> They perform a POST request to the Keycloak SAML endpoint of the configured
> client, leading to a HTTP 405 error.
>
>
> The fact that they perform a POST to load the Keycloak login page is
> discussable, but how to counter this behaviour on Keycloak client's
> configuration side?
>
>
> Thank you in advance for your guidance.
>
>
> Cheers,
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev<
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.jboss.org_mailman_listinfo_keycloak-2Ddev&d=DwMFaQ&c=8G9fL-FODW3I-fESjFvsaw&r=BhQIS0sALperhBRJRNzxM3Ax9g4k8rNIuzNmM35TMgE&m=72xRw4Glpv6dKcdXHkNdMWSdUO2SnV-al0GTcZGzDiE&s=_Q5d2-MpbXWlpKgBC0-nBHGbv6HpUu6ucLAhQjfFK-o&e=
> >
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From Sebastian.Loesch at governikus.de  Tue Sep  4 02:40:56 2018
From: Sebastian.Loesch at governikus.de (=?utf-8?B?TMO2c2NoLCBTZWJhc3RpYW4=?=)
Date: Tue, 4 Sep 2018 06:40:56 +0000
Subject: [keycloak-dev] SCIM v2 support
In-Reply-To: <CAJgngAd8-V++LvJijHR4a7nOk-KyqOg6qCA3HaG=uR-fYC0q9A@mail.gmail.com>
References: <93f62b8abc994e88835f314118fb9f4d@BOSKGEXC01.boskg.local>
	<CAJgngAd8-V++LvJijHR4a7nOk-KyqOg6qCA3HaG=uR-fYC0q9A@mail.gmail.com>
Message-ID: <abb8e9ae749342ad811932702785ea8c@BOSKGEXC01.boskg.local>

Hello Stian,



thank you for your input. Storing additional SCIM attributes as key/value pairs in the USER_ATTRIBUTE table has two drawbacks:

-          Storing of multivalued complex attributes becomes difficult. E.g. the SCIM phone number type has 4 attributes ?value?, ?display?, ?type? and ?primary? and each user may have many phone number objects.

-          Searching for users by attributes becomes difficult und maybe has a bad performance because of the SQL.

Do you still think that?s the preferred way to go?



Best regards,

Sebastian



Von: Stian Thorgersen <sthorger at redhat.com>
Gesendet: Montag, 3. September 2018 12:21
An: L?sch, Sebastian <Sebastian.Loesch at governikus.de>
Cc: keycloak-dev <keycloak-dev at lists.jboss.org>
Betreff: Re: [keycloak-dev] SCIM v2 support



Adding additional attributes to user entity is probably not the way to go. Rather, it would be better for backwards compatibility to simply use generic key/value attributes which the user entity already has.



Implementing the SCIM endpoints is probably pretty straightforward. Most of the work will probably be down to testing and documentation.

On Wed, 29 Aug 2018 at 13:28, L?sch, Sebastian <Sebastian.Loesch at governikus.de<mailto:Sebastian.Loesch at governikus.de>> wrote:

   Hello,



   in a customer project we use keycloak and need a SCIM (System for
   Cross-domain Identity Management) API.

   Currently we write a wrapper API and a custom endpoint providing the SCIM
   functionality. We wrote a extension of the UserEntity, UserModel and an
   extension of the JpaUserProvider.

   This strategy seems not ideal and the nicest way is to add this extensions
   to Keycloak. This is already suggested in
   https://issues.jboss.org/browse/KEYCLOAK-2537

   Is anybody out there who can guide me, what coding would be necessary to
   contribute the SCIM functionality?



   Currently I think we have to:

   -          extend the UserEntity with all SCIM attributes. This will result
   in additional tables/entities for complex attributes e.g. Address, Name,
   Email

   -          extend the UserModel to povide the additional attributes

   -          implement the new SCIM endpoint /Users

   -          make the additional attributes available via Admin REST API
   /users

   -          extend views to be able to edit SCIM user attributes using the
   web ui

   -          ?

   -          All the above again for the Groups endpoint?



   This also seem to be major changes. To big for one Pull Request. How do you
   like to handle this?



   Best regards,

   Sebastian





   _______________________________________________
   keycloak-dev mailing list
   keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>
   https://lists.jboss.org/mailman/listinfo/keycloak-dev


From Sebastian.Schuster at bosch-si.com  Tue Sep  4 03:50:34 2018
From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST-CSS/BSV-OS))
Date: Tue, 4 Sep 2018 07:50:34 +0000
Subject: [keycloak-dev] Potential performance regression between
 3.4.3.Final and 4.3.0.Final
In-Reply-To: <CAMvXD=EFbLi2uSw5UkGOGbqo2za9CEvFQoEnfWXOeQfAdyjHjQ@mail.gmail.com>
References: <CAK-7U1j0kc=19-UBp=c64ykT8j1ho39HwjVydEEoWdO9L3Q+ZQ@mail.gmail.com>
	<CAJgngAeka1tuAw0VhiAj3RfOLagwYt_dgi=B8sgzcBQFVmfmpw@mail.gmail.com>
	<d8b5a47322e34d879347af8c646e90b9@bosch-si.com>
	<CAK-7U1g2OKZZkrXbbWy5kQ1r+CDNHgZtbYuCoWkLEiyXL4D_-g@mail.gmail.com>
	<CAJgngAffB30sXjUjQdyeGCQ=F19OHTvVQ9+RTevxBJObcnfEhg@mail.gmail.com>
	<CAK-7U1hEtDTnm4Lv6hkkQGkCCwjALitsBNT5O+m94oygX8CWVw@mail.gmail.com>
	<CAMvXD=EaZkgJ2Ur0A=MDDLtAm1K=hjTjW2ehgC9CXkiW-co7ow@mail.gmail.com>
	<3de878395d3f4e4f8dd5e872d9e41e98@bosch-si.com>
	<CAK-7U1jZpYzRTGXqSW=44AvhHN+j8dZKFNGL6MxccGO=UB66wA@mail.gmail.com>
	<bce51bcfb6834730b8ba28c1d9e583f6@bosch-si.com>
	<CAMvXD=EMJt8u5HSCtq7rGvYd3W0dBza1Kg0yFWFE_uY0DO-nMw@mail.gmail.com>
	<CAMvXD=EFbLi2uSw5UkGOGbqo2za9CEvFQoEnfWXOeQfAdyjHjQ@mail.gmail.com>
Message-ID: <3f90f66ead604b27ae0862e84a1fd224@bosch-si.com>

Hi Hynek,

I did a little testing. In my setup I have a local MS SQL DB with 170.495 users and 18.663.024 user attributes. That?s in the order of our targeted production setup.

Without your change, listing users basically times out, I never get a response.
With your change not fetching the user attributes I get an answer after 150 ms.

I also tested what happens if I change the fetchtype to ?select? of all one-to-many relations in UserEntity with a batch size of 20 (just an initial guess as that?s the default page size in the UI).
Then I get the list of users in 1100ms instead of timing out (after setting briefRepresentation to false).

I read a little bit in the Hibernate documentation and they wrote that fetchmode.subselect loads all entities into the session and only makes sense when nearly all entities should be loaded anyways. I?d guess that?s mostly not true for Users in Keycloak. Also not for Resources, the only other place I found a subselect relation.

I would conclude going back to fetchtype.select with a batch size of 20 is definitely superior to the subselect that?s currently used, independently of your change.
It?s just 6 lines of code change. ? I could prepare a pull request for that but I have to go through some nasty processes on our side so that might take a while?

Best regards,
Sebastian

Mit freundlichen Gr??en / Best regards

Dr.-Ing. Sebastian Schuster

Engineering and Support (INST/ESY1)
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn



From: Hynek Mlnarik <hmlnarik at redhat.com>
Sent: Freitag, 31. August 2018 18:38
To: Thomas Darimont <thomas.darimont at googlemail.com>
Cc: Schuster Sebastian (INST/ESY1) <Sebastian.Schuster at bosch-si.com>; Stian Thorgersen <stian at redhat.com>; keycloak-dev <keycloak-dev at lists.jboss.org>
Subject: Re: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final

Hi Thomas,

the issue with repeating a query should be addressed by [1]. Out of curiosity, I wonder whether you'd have some spare time to test this change in your environment?

Thanks

--Hynek

[1] https://github.com/keycloak/keycloak/pull/5525



On Thu, Aug 30, 2018 at 12:58 PM Hynek Mlnarik <hmlnarik at redhat.com<mailto:hmlnarik at redhat.com>> wrote:
Hi Sebastian, Thomas,

I agree that this query is bad performance-wise. It is caused inability of database to easily limit rows in the resultset of query that joins users and attributes (and other tables) if there might are several attributes per user while we want to return exactly n users (the LIMIT/OFFSET won't work here for parent table due to joining with child records). Hibernate thus resorts to its own filtering of the full resultset. That is one option, the other is having subsequent selects and limiting batch size. Both approaches have pros and cons, and actually the subsequent selects were changed to subselects precisely because of performance. Looks like we might need to revert back to the select approach and tweak batch size. That will require some time for optimizations.

For now, the most painful issue with user list seems resolved and thus I'm merging the PR. There will be subsequent work though on further optimizations to user / user list retrieval, that relates to both your suggestion and Thomas's comment (that relates to authz resource server store caching). Thank you both for your help!

--Hynek


On Thu, Aug 30, 2018 at 12:06 PM Schuster Sebastian (INST/ESY1) <Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>> wrote:
Hi Thomas,

Your query is the same as before, it is the initial retrieval of the user entity. The query I am referring to is the one loading the user attributes after the user entity has been fetched as part of ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel realm, UserModel user). It just looks wrong to me that all users are fetched when only the attributes of a single user are needed. However, with Hynek?s change, the user attributes are no longer fetched because only a BriefRepresentation is requested, that?s why it is faster now. I still think a full representation should also be faster and not fetch all users from a realm.

Best regards,
Sebastian

Mit freundlichen Gr??en / Best regards

Dr.-Ing. Sebastian Schuster

Engineering and Support (INST/ESY1)
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn


From: Thomas Darimont <thomas.darimont at googlemail.com<mailto:thomas.darimont at googlemail.com>>
Sent: Mittwoch, 29. August 2018 22:55
To: Schuster Sebastian (INST/ESY1) <Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>>
Cc: Hynek Mlnarik <hmlnarik at redhat.com<mailto:hmlnarik at redhat.com>>; Stian Thorgersen <stian at redhat.com<mailto:stian at redhat.com>>; keycloak-dev <keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>>
Subject: Re: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final

Hello Hynek et al.,

I gave your PR 5517 a quick spin and tested the user listing in the admin-console on my Dell XPS (3.4GHz, 32g) laptop with local PostgreSQL backend.
I can confirm that the response times for loading a single page in the user-listing are much better now with your fix:
In a test realm with 10k users, loading a single page takes ~15ms compared to 8 seconds (!) before.

The new query that is now generated whilst browsing the user list looks like this:

Hibernate:
    select
        userentity0_.ID as ID1_75_,
        userentity0_.CREATED_TIMESTAMP as CREATED_2_75_,
        userentity0_.EMAIL as EMAIL3_75_,
        userentity0_.EMAIL_CONSTRAINT as EMAIL_CO4_75_,
        userentity0_.EMAIL_VERIFIED as EMAIL_VE5_75_,
        userentity0_.ENABLED as ENABLED6_75_,
        userentity0_.FEDERATION_LINK as FEDERATI7_75_,
        userentity0_.FIRST_NAME as FIRST_NA8_75_,
        userentity0_.LAST_NAME as LAST_NAM9_75_,
        userentity0_.NOT_BEFORE as NOT_BEF10_75_,
        userentity0_.REALM_ID as REALM_I11_75_,
        userentity0_.SERVICE_ACCOUNT_CLIENT_LINK as SERVICE12_75_,
        userentity0_.USERNAME as USERNAM13_75_
    from
        USER_ENTITY userentity0_
    where
        userentity0_.REALM_ID=?
        and (
            userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null
        )
    order by
        userentity0_.USERNAME limit ? offset ?

The query looks reasonable with appropriate limits applied.

However, it seems that policies are fetched for each returned user via a dedicated query, e.g.:

    select
        resourcese0_.ID as ID1_64_0_,
        resourcese0_.ALLOW_RS_REMOTE_MGMT as ALLOW_RS2_64_0_,
        resourcese0_.POLICY_ENFORCE_MODE as POLICY_E3_64_0_
    from
        RESOURCE_SERVER resourcese0_
    where
        resourcese0_.ID=?

Would it be possible to load those in one batch?

Cheers,
Thomas

Am Mi., 29. Aug. 2018 um 17:19 Uhr schrieb Schuster Sebastian (INST/ESY1) <Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>>:
Thanks Hynek, I?ll test this. I also tracked the issue down to the ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel realm, UserModel user) method. However, what I am also seeing is when the method is running, it is emitting SQL queries that are not scoped down to the user id of the user being transformed. E.g. for the user attributes of a user the following query is generated:

select
        attributes0_.USER_ID as USER_ID4_72_1_,
        attributes0_.ID as ID1_72_1_,
        attributes0_.ID as ID1_72_0_,
        attributes0_.NAME as NAME2_72_0_,
        attributes0_.USER_ID as USER_ID4_72_0_,
        attributes0_.VALUE as VALUE3_72_0_
    from
        USER_ATTRIBUTE attributes0_
    where
        attributes0_.USER_ID in (
            select
                userentity0_.ID
            from
                USER_ENTITY userentity0_
            where
                userentity0_.REALM_ID=?
                and (
                    userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null
                )
        )

The subselect returns all real users from the realm. It is neither paged nor bound to a specific user id. The same happens in the above method for credentials and required actions. I am not a JPA/Hibernate expert so I might be missing something here?

Best regards,
Sebastian

Mit freundlichen Gr??en / Best regards

Dr.-Ing. Sebastian Schuster

Engineering and Support (INST/ESY1)
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn

From: Hynek Mlnarik <hmlnarik at redhat.com<mailto:hmlnarik at redhat.com>>
Sent: Mittwoch, 29. August 2018 16:35
To: Thomas Darimont <thomas.darimont at googlemail.com<mailto:thomas.darimont at googlemail.com>>
Cc: Stian Thorgersen <stian at redhat.com<mailto:stian at redhat.com>>; keycloak-dev <keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>>; Schuster Sebastian (INST/ESY1) <Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>>
Subject: Re: [keycloak-dev] Potential performance regression between 3.4.3.Final and 4.3.0.Final

Thanks Thomas and all,

great help with reproducing the issue at least for admin user list endpoint. Could you please test [1] that it indeed fixes the issue? It was caused obtaining too much user details when unnecessary (for user list, only few fields are necessary).

As for increased memory requirements, these might be unrelated to this fix, and we'll keep an eye on this.

--Hynek

[1] https://github.com/keycloak/keycloak/pull/5517



On Mon, Aug 27, 2018 at 4:48 PM Thomas Darimont <thomas.darimont at googlemail.com<mailto:thomas.darimont at googlemail.com>> wrote:
Hello,

FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150

Cheers,
Thomas

Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen <
sthorger at redhat.com<mailto:sthorger at redhat.com>>:

> Can you open a bug for this in JIRA please and include as much details as
> you can.
>
> On Thu, 23 Aug 2018 at 15:45, Thomas Darimont <
> thomas.darimont at googlemail.com<mailto:thomas.darimont at googlemail.com>> wrote:
>
>> Hi Sebastian,
>>
>> thanks for investigating! Unfortunately, I didn't have time yet to look
>> further into the problem, but I also observed the slow attributes handling.
>> Btw. we have around 100k users and ~700k attributes. The
>> Entity-Attribute-Value data model takes it's toll here...
>>
>> @Stian so far I've only observed this when scrolling through the user
>> overview in the admin-console, which also triggers
>> the mentioned spike in heap usage.
>>
>> Cheers,
>> Thomas
>>
>> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian (INST/ESY1)
>> <Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>>:
>>
>>> Btw. we also have 40K user attributes...
>>>
>>> Best regards,
>>> Sebastian
>>>
>>> Mit freundlichen Gr??en / Best regards
>>>
>>> Dr.-Ing.  Sebastian Schuster
>>>
>>> Engineering and Support (INST/ESY1)
>>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
>>> GERMANY | www.bosch-si.com<http://www.bosch-si.com>
>>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
>>> Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>
>>>
>>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
>>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung:
>>> Dr. Stefan Ferber, Michael Hahn
>>>
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: keycloak-dev-bounces at lists.jboss.org<mailto:keycloak-dev-bounces at lists.jboss.org> <
>>> keycloak-dev-bounces at lists.jboss.org<mailto:keycloak-dev-bounces at lists.jboss.org>> On Behalf Of Stian Thorgersen
>>> Sent: Mittwoch, 22. August 2018 21:25
>>> To: Thomas Darimont <thomas.darimont at googlemail.com<mailto:thomas.darimont at googlemail.com>>
>>> Cc: keycloak-dev <keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>>
>>> Subject: Re: [keycloak-dev] Potential performance regression between
>>> 3.4.3.Final and 4.3.0.Final
>>>
>>> That's quite worrying. Is it limited to browsing users through the admin
>>> console or are you seeing bad behaviour elsewhere? Same question applies to
>>> the heap. My best bet here is that it has that something has changed around
>>> user querying/caching.
>>>
>>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont <
>>> thomas.darimont at googlemail.com<mailto:thomas.darimont at googlemail.com>> wrote:
>>>
>>> > Hello Keycloak team,
>>> >
>>> > has anyone encountered some performance issues after upgrading 3.4.3
>>> > to 4.x (4.3.0)?
>>> >
>>> > Today I noticed a performance regression while preparing an upgrade
>>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment.
>>> >
>>> > In our test environment, we have around ~100k test users stored in a
>>> > postgres-backed database. When we started the server with the new
>>> > Keycloak version, the migration went through, and everything looked
>>> > fine at first, but when we tried to browse the list of users via the
>>> > admin-console, we noticed that the CPU and memory consumption of the
>>> > server increased significantly, up to a point where Keycloak crashed
>>> with an OOME.
>>> >
>>> > All previous Keycloak versions including 3.4.3 were very modest with
>>> > their memory requirements and quite happy with ~1g heap.
>>> > However, that seems to have changed in Keycloak 4.3.0 - there we
>>> > needed at least 4g to prevent Keycloak from crashing with an OOME.
>>> > Furthermore, we noticed that the response times for browsing the
>>> > paginated user view increased significantly as well:
>>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In
>>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation
>>> > takes
>>> > ~7 seconds for a test realm with just 10k users.
>>> >
>>> > In the test realm with 100k users, the time to load a single page in
>>> > the users listing was 66 seconds for version 4.3.0, on average -
>>> > compared to quite stable 80ms in 3.4.3.
>>> >
>>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5
>>> > seconds for 100k users, so I assume the processing logic in Keycloak
>>> > is the culprit.
>>> >
>>> > The problem of long load-times can be reproduced with the Keycloak
>>> > docker images and the in-memory database. I also created a small
>>> > example project that creates some users with just a few attributes in
>>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres
>>> > database to reproduce the problem.
>>> > https://github.com/thomasdarimont/kc-user-regression-tester
>>> >
>>> > Cheers,
>>> > Thomas
>>> > _______________________________________________
>>> > keycloak-dev mailing list
>>> > keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>
>>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> >
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>
_______________________________________________
keycloak-dev mailing list
keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-dev

From hmlnarik at redhat.com  Tue Sep  4 04:54:48 2018
From: hmlnarik at redhat.com (Hynek Mlnarik)
Date: Tue, 4 Sep 2018 10:54:48 +0200
Subject: [keycloak-dev] Potential performance regression between
 3.4.3.Final and 4.3.0.Final
In-Reply-To: <3f90f66ead604b27ae0862e84a1fd224@bosch-si.com>
References: <CAK-7U1j0kc=19-UBp=c64ykT8j1ho39HwjVydEEoWdO9L3Q+ZQ@mail.gmail.com>
	<CAJgngAeka1tuAw0VhiAj3RfOLagwYt_dgi=B8sgzcBQFVmfmpw@mail.gmail.com>
	<d8b5a47322e34d879347af8c646e90b9@bosch-si.com>
	<CAK-7U1g2OKZZkrXbbWy5kQ1r+CDNHgZtbYuCoWkLEiyXL4D_-g@mail.gmail.com>
	<CAJgngAffB30sXjUjQdyeGCQ=F19OHTvVQ9+RTevxBJObcnfEhg@mail.gmail.com>
	<CAK-7U1hEtDTnm4Lv6hkkQGkCCwjALitsBNT5O+m94oygX8CWVw@mail.gmail.com>
	<CAMvXD=EaZkgJ2Ur0A=MDDLtAm1K=hjTjW2ehgC9CXkiW-co7ow@mail.gmail.com>
	<3de878395d3f4e4f8dd5e872d9e41e98@bosch-si.com>
	<CAK-7U1jZpYzRTGXqSW=44AvhHN+j8dZKFNGL6MxccGO=UB66wA@mail.gmail.com>
	<bce51bcfb6834730b8ba28c1d9e583f6@bosch-si.com>
	<CAMvXD=EMJt8u5HSCtq7rGvYd3W0dBza1Kg0yFWFE_uY0DO-nMw@mail.gmail.com>
	<CAMvXD=EFbLi2uSw5UkGOGbqo2za9CEvFQoEnfWXOeQfAdyjHjQ@mail.gmail.com>
	<3f90f66ead604b27ae0862e84a1fd224@bosch-si.com>
Message-ID: <CAMvXD=F9-yLFNxArHYP9Lf08R4Tc9H=pzs+ECZ5GJGEytK=NiA@mail.gmail.com>

Hi Sebastian,

thanks for checking on your dataset, really appreciated!

PR would be welcome, I believe batch size of 20 makes sense at least for
this PR, we can optimize further if if that is not sufficient. Please refer
to KEYCLOAK-8222 in the commit.

--Hynek

On Tue, Sep 4, 2018 at 9:51 AM Schuster Sebastian (INST-CSS/BSV-OS) <
Sebastian.Schuster at bosch-si.com> wrote:

> Hi Hynek,
>
>
>
> I did a little testing. In my setup I have a local MS SQL DB with 170.495
> users and 18.663.024 user attributes. That?s in the order of our targeted
> production setup.
>
>
>
> Without your change, listing users basically times out, I never get a
> response.
>
> With your change not fetching the user attributes I get an answer after
> 150 ms.
>
>
>
> I also tested what happens if I change the fetchtype to ?select? of all
> one-to-many relations in UserEntity with a batch size of 20 (just an
> initial guess as that?s the default page size in the UI).
>
> Then I get the list of users in 1100ms instead of timing out (after
> setting briefRepresentation to false).
>
>
>
> I read a little bit in the Hibernate documentation and they wrote that
> fetchmode.subselect loads all entities into the session and only makes
> sense when nearly all entities should be loaded anyways. I?d guess that?s
> mostly not true for Users in Keycloak. Also not for Resources, the only
> other place I found a subselect relation.
>
>
>
> I would conclude going back to fetchtype.select with a batch size of 20 is
> definitely superior to the subselect that?s currently used, independently
> of your change.
>
> It?s just 6 lines of code change. J I could prepare a pull request for
> that but I have to go through some nasty processes on our side so that
> might take a while?
>
>
>
> Best regards,
>
> Sebastian
>
>
>
> Mit freundlichen Gr??en / Best regards
>
>
> *Dr.-Ing. Sebastian Schuster *
> Engineering and Support (INST/ESY1)
> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> GERMANY | www.bosch-si.com
> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
> Sebastian.Schuster at bosch-si.com
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr.
> Stefan Ferber, Michael Hahn
>
>
>
> *From:* Hynek Mlnarik <hmlnarik at redhat.com>
> *Sent:* Freitag, 31. August 2018 18:38
> *To:* Thomas Darimont <thomas.darimont at googlemail.com>
> *Cc:* Schuster Sebastian (INST/ESY1) <Sebastian.Schuster at bosch-si.com>;
> Stian Thorgersen <stian at redhat.com>; keycloak-dev <
> keycloak-dev at lists.jboss.org>
> *Subject:* Re: [keycloak-dev] Potential performance regression between
> 3.4.3.Final and 4.3.0.Final
>
>
>
> Hi Thomas,
>
>
>
> the issue with repeating a query should be addressed by [1]. Out of
> curiosity, I wonder whether you'd have some spare time to test this change
> in your environment?
>
>
>
> Thanks
>
>
>
> --Hynek
>
>
>
> [1] https://github.com/keycloak/keycloak/pull/5525
>
>
>
>
>
>
>
> On Thu, Aug 30, 2018 at 12:58 PM Hynek Mlnarik <hmlnarik at redhat.com>
> wrote:
>
> Hi Sebastian, Thomas,
>
>
>
> I agree that this query is bad performance-wise. It is caused inability of
> database to easily limit rows in the resultset of query that joins users
> and attributes (and other tables) if there might are several attributes per
> user while we want to return exactly n users (the LIMIT/OFFSET won't work
> here for parent table due to joining with child records). Hibernate thus
> resorts to its own filtering of the full resultset. That is one option, the
> other is having subsequent selects and limiting batch size. Both approaches
> have pros and cons, and actually the subsequent selects were changed to
> subselects precisely because of performance. Looks like we might need to
> revert back to the select approach and tweak batch size. That will require
> some time for optimizations.
>
>
>
> For now, the most painful issue with user list seems resolved and thus I'm
> merging the PR. There will be subsequent work though on further
> optimizations to user / user list retrieval, that relates to both your
> suggestion and Thomas's comment (that relates to authz resource server
> store caching). Thank you both for your help!
>
>
>
> --Hynek
>
>
>
>
>
> On Thu, Aug 30, 2018 at 12:06 PM Schuster Sebastian (INST/ESY1) <
> Sebastian.Schuster at bosch-si.com> wrote:
>
> Hi Thomas,
>
>
>
> Your query is the same as before, it is the initial retrieval of the user
> entity. The query I am referring to is the one loading the user attributes
> after the user entity has been fetched as part of
> ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel
> realm, UserModel user). It just looks wrong to me that all users are
> fetched when only the attributes of a single user are needed. However, with
> Hynek?s change, the user attributes are no longer fetched because only a
> BriefRepresentation is requested, that?s why it is faster now. I still
> think a full representation should also be faster and not fetch all users
> from a realm.
>
>
>
> Best regards,
>
> Sebastian
>
>
>
> Mit freundlichen Gr??en / Best regards
>
>
> *Dr.-Ing. Sebastian Schuster *
> Engineering and Support (INST/ESY1)
> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> GERMANY | www.bosch-si.com
> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
> Sebastian.Schuster at bosch-si.com
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr.
> Stefan Ferber, Michael Hahn
>
>
> *From:* Thomas Darimont <thomas.darimont at googlemail.com>
> *Sent:* Mittwoch, 29. August 2018 22:55
> *To:* Schuster Sebastian (INST/ESY1) <Sebastian.Schuster at bosch-si.com>
> *Cc:* Hynek Mlnarik <hmlnarik at redhat.com>; Stian Thorgersen <
> stian at redhat.com>; keycloak-dev <keycloak-dev at lists.jboss.org>
> *Subject:* Re: [keycloak-dev] Potential performance regression between
> 3.4.3.Final and 4.3.0.Final
>
>
>
> Hello Hynek et al.,
>
>
>
> I gave your PR 5517 a quick spin and tested the user listing in the
> admin-console on my Dell XPS (3.4GHz, 32g) laptop with local PostgreSQL
> backend.
>
> I can confirm that the response times for loading a single page in the
> user-listing are much better now with your fix:
>
> In a test realm with 10k users, loading a single page takes ~15ms compared
> to 8 seconds (!) before.
>
>
>
> The new query that is now generated whilst browsing the user list looks
> like this:
>
>
>
> Hibernate:
>
>     select
>
>         userentity0_.ID as ID1_75_,
>
>         userentity0_.CREATED_TIMESTAMP as CREATED_2_75_,
>
>         userentity0_.EMAIL as EMAIL3_75_,
>
>         userentity0_.EMAIL_CONSTRAINT as EMAIL_CO4_75_,
>
>         userentity0_.EMAIL_VERIFIED as EMAIL_VE5_75_,
>
>         userentity0_.ENABLED as ENABLED6_75_,
>
>         userentity0_.FEDERATION_LINK as FEDERATI7_75_,
>
>         userentity0_.FIRST_NAME as FIRST_NA8_75_,
>
>         userentity0_.LAST_NAME as LAST_NAM9_75_,
>
>         userentity0_.NOT_BEFORE as NOT_BEF10_75_,
>
>         userentity0_.REALM_ID as REALM_I11_75_,
>
>         userentity0_.SERVICE_ACCOUNT_CLIENT_LINK as SERVICE12_75_,
>
>         userentity0_.USERNAME as USERNAM13_75_
>
>     from
>
>         USER_ENTITY userentity0_
>
>     where
>
>         userentity0_.REALM_ID=?
>
>         and (
>
>             userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null
>
>         )
>
>     order by
>
>         userentity0_.USERNAME limit ? offset ?
>
>
>
> The query looks reasonable with appropriate limits applied.
>
>
>
> However, it seems that policies are fetched for each returned user via a
> dedicated query, e.g.:
>
>
>
>     select
>
>         resourcese0_.ID as ID1_64_0_,
>
>         resourcese0_.ALLOW_RS_REMOTE_MGMT as ALLOW_RS2_64_0_,
>
>         resourcese0_.POLICY_ENFORCE_MODE as POLICY_E3_64_0_
>
>     from
>
>         RESOURCE_SERVER resourcese0_
>
>     where
>
>         resourcese0_.ID=?
>
>
>
> Would it be possible to load those in one batch?
>
>
>
> Cheers,
>
> Thomas
>
>
>
> Am Mi., 29. Aug. 2018 um 17:19 Uhr schrieb Schuster Sebastian (INST/ESY1) <
> Sebastian.Schuster at bosch-si.com>:
>
> Thanks Hynek, I?ll test this. I also tracked the issue down to the
> ModelToRepresentation.toRepresentation(KeycloakSession session, RealmModel
> realm, UserModel user) method. However, what I am also seeing is when the
> method is running, it is emitting SQL queries that are not scoped down to
> the user id of the user being transformed. E.g. for the user attributes of
> a user the following query is generated:
>
>
>
> select
>
>         attributes0_.USER_ID as USER_ID4_72_1_,
>
>         attributes0_.ID as ID1_72_1_,
>
>         attributes0_.ID as ID1_72_0_,
>
>         attributes0_.NAME as NAME2_72_0_,
>
>         attributes0_.USER_ID as USER_ID4_72_0_,
>
>         attributes0_.VALUE as VALUE3_72_0_
>
>     from
>
>         USER_ATTRIBUTE attributes0_
>
>     where
>
>         attributes0_.USER_ID in (
>
>             select
>
>                 userentity0_.ID
>
>             from
>
>                 USER_ENTITY userentity0_
>
>             where
>
>                 userentity0_.REALM_ID=?
>
>                 and (
>
>                     userentity0_.SERVICE_ACCOUNT_CLIENT_LINK is null
>
>                 )
>
>         )
>
>
>
> The subselect returns all real users from the realm. It is neither paged
> nor bound to a specific user id. The same happens in the above method for
> credentials and required actions. I am not a JPA/Hibernate expert so I
> might be missing something here?
>
>
>
> Best regards,
>
> Sebastian
>
>
>
> Mit freundlichen Gr??en / Best regards
>
>
> *Dr.-Ing. Sebastian Schuster *
> Engineering and Support (INST/ESY1)
> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> GERMANY | www.bosch-si.com
> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
> Sebastian.Schuster at bosch-si.com
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr.
> Stefan Ferber, Michael Hahn
>
> *From:* Hynek Mlnarik <hmlnarik at redhat.com>
> *Sent:* Mittwoch, 29. August 2018 16:35
> *To:* Thomas Darimont <thomas.darimont at googlemail.com>
> *Cc:* Stian Thorgersen <stian at redhat.com>; keycloak-dev <
> keycloak-dev at lists.jboss.org>; Schuster Sebastian (INST/ESY1) <
> Sebastian.Schuster at bosch-si.com>
> *Subject:* Re: [keycloak-dev] Potential performance regression between
> 3.4.3.Final and 4.3.0.Final
>
>
>
> Thanks Thomas and all,
>
>
>
> great help with reproducing the issue at least for admin user list
> endpoint. Could you please test [1] that it indeed fixes the issue? It was
> caused obtaining too much user details when unnecessary (for user list,
> only few fields are necessary).
>
>
>
> As for increased memory requirements, these might be unrelated to this
> fix, and we'll keep an eye on this.
>
>
>
> --Hynek
>
>
>
> [1] https://github.com/keycloak/keycloak/pull/5517
>
>
>
>
>
>
>
> On Mon, Aug 27, 2018 at 4:48 PM Thomas Darimont <
> thomas.darimont at googlemail.com> wrote:
>
> Hello,
>
> FYI I created https://issues.jboss.org/browse/KEYCLOAK-8150
>
> Cheers,
> Thomas
>
> Am Mo., 27. Aug. 2018 um 13:33 Uhr schrieb Stian Thorgersen <
> sthorger at redhat.com>:
>
> > Can you open a bug for this in JIRA please and include as much details as
> > you can.
> >
> > On Thu, 23 Aug 2018 at 15:45, Thomas Darimont <
> > thomas.darimont at googlemail.com> wrote:
> >
> >> Hi Sebastian,
> >>
> >> thanks for investigating! Unfortunately, I didn't have time yet to look
> >> further into the problem, but I also observed the slow attributes
> handling.
> >> Btw. we have around 100k users and ~700k attributes. The
> >> Entity-Attribute-Value data model takes it's toll here...
> >>
> >> @Stian so far I've only observed this when scrolling through the user
> >> overview in the admin-console, which also triggers
> >> the mentioned spike in heap usage.
> >>
> >> Cheers,
> >> Thomas
> >>
> >> Am Do., 23. Aug. 2018 um 15:22 Uhr schrieb Schuster Sebastian
> (INST/ESY1)
> >> <Sebastian.Schuster at bosch-si.com>:
> >>
> >>> Btw. we also have 40K user attributes...
> >>>
> >>> Best regards,
> >>> Sebastian
> >>>
> >>> Mit freundlichen Gr??en / Best regards
> >>>
> >>> Dr.-Ing.  Sebastian Schuster
> >>>
> >>> Engineering and Support (INST/ESY1)
> >>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> >>> GERMANY | www.bosch-si.com
> >>> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
> >>> Sebastian.Schuster at bosch-si.com
> >>>
> >>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> >>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung:
> >>> Dr. Stefan Ferber, Michael Hahn
> >>>
> >>>
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: keycloak-dev-bounces at lists.jboss.org <
> >>> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Stian Thorgersen
> >>> Sent: Mittwoch, 22. August 2018 21:25
> >>> To: Thomas Darimont <thomas.darimont at googlemail.com>
> >>> Cc: keycloak-dev <keycloak-dev at lists.jboss.org>
> >>> Subject: Re: [keycloak-dev] Potential performance regression between
> >>> 3.4.3.Final and 4.3.0.Final
> >>>
> >>> That's quite worrying. Is it limited to browsing users through the
> admin
> >>> console or are you seeing bad behaviour elsewhere? Same question
> applies to
> >>> the heap. My best bet here is that it has that something has changed
> around
> >>> user querying/caching.
> >>>
> >>> On Mon, 20 Aug 2018 at 19:18, Thomas Darimont <
> >>> thomas.darimont at googlemail.com> wrote:
> >>>
> >>> > Hello Keycloak team,
> >>> >
> >>> > has anyone encountered some performance issues after upgrading 3.4.3
> >>> > to 4.x (4.3.0)?
> >>> >
> >>> > Today I noticed a performance regression while preparing an upgrade
> >>> > from Keycloak 3.4.3.Final to 4.3.0.Final in our staging environment.
> >>> >
> >>> > In our test environment, we have around ~100k test users stored in a
> >>> > postgres-backed database. When we started the server with the new
> >>> > Keycloak version, the migration went through, and everything looked
> >>> > fine at first, but when we tried to browse the list of users via the
> >>> > admin-console, we noticed that the CPU and memory consumption of the
> >>> > server increased significantly, up to a point where Keycloak crashed
> >>> with an OOME.
> >>> >
> >>> > All previous Keycloak versions including 3.4.3 were very modest with
> >>> > their memory requirements and quite happy with ~1g heap.
> >>> > However, that seems to have changed in Keycloak 4.3.0 - there we
> >>> > needed at least 4g to prevent Keycloak from crashing with an OOME.
> >>> > Furthermore, we noticed that the response times for browsing the
> >>> > paginated user view increased significantly as well:
> >>> > In Keycloak 3.4.3 the average time to load a user page is ~80ms. In
> >>> > Keycloak 4.3.0 (and older versions >= 4.0.0.Beta1) the same operation
> >>> > takes
> >>> > ~7 seconds for a test realm with just 10k users.
> >>> >
> >>> > In the test realm with 100k users, the time to load a single page in
> >>> > the users listing was 66 seconds for version 4.3.0, on average -
> >>> > compared to quite stable 80ms in 3.4.3.
> >>> >
> >>> > The database query that is executed by Keycloak 4.3.0 runs in ~1.5
> >>> > seconds for 100k users, so I assume the processing logic in Keycloak
> >>> > is the culprit.
> >>> >
> >>> > The problem of long load-times can be reproduced with the Keycloak
> >>> > docker images and the in-memory database. I also created a small
> >>> > example project that creates some users with just a few attributes in
> >>> > a docker based 3.4.3 and 4.3.0 Keycloak environment with a Postgres
> >>> > database to reproduce the problem.
> >>> > https://github.com/thomasdarimont/kc-user-regression-tester
> >>> >
> >>> > Cheers,
> >>> > Thomas
> >>> > _______________________________________________
> >>> > keycloak-dev mailing list
> >>> > keycloak-dev at lists.jboss.org
> >>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>> >
> >>> _______________________________________________
> >>> keycloak-dev mailing list
> >>> keycloak-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>>
> >>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>

From sthorger at redhat.com  Tue Sep  4 06:15:06 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Tue, 4 Sep 2018 12:15:06 +0200
Subject: [keycloak-dev] SCIM v2 support
In-Reply-To: <abb8e9ae749342ad811932702785ea8c@BOSKGEXC01.boskg.local>
References: <93f62b8abc994e88835f314118fb9f4d@BOSKGEXC01.boskg.local>
	<CAJgngAd8-V++LvJijHR4a7nOk-KyqOg6qCA3HaG=uR-fYC0q9A@mail.gmail.com>
	<abb8e9ae749342ad811932702785ea8c@BOSKGEXC01.boskg.local>
Message-ID: <CAJgngAeg3Pumtd=yemA2rFvtUAr0GLteWqdHFV4tcMe74akGLA@mail.gmail.com>

It's not ideal, but adding columns for every additional attribute is also
far from ideal. Then there's the issue with backwards compatibility with
regards to user storage providers as that is a supported SPI and the API
needs to remain backwards compatible.

An complicating factor for SCIM will be user storage providers. Attributes,
queries, etc. should all work when there are user storage providers in the
realm. These could be LDAP or a custom implementation.

Queries on attributes shouldn't be to bad as long as the query is written
properly.

On Tue, 4 Sep 2018 at 08:41, L?sch, Sebastian <
Sebastian.Loesch at governikus.de> wrote:

> Hello Stian,
>
>
>
> thank you for your input. Storing additional SCIM attributes as key/value
> pairs in the USER_ATTRIBUTE table has two drawbacks:
>
> -          Storing of multivalued complex attributes becomes difficult.
> E.g. the SCIM phone number type has 4 attributes ?value?, ?display?, ?type?
> and ?primary? and each user may have many phone number objects.
>
> -          Searching for users by attributes becomes difficult und maybe
> has a bad performance because of the SQL.
>
> Do you still think that?s the preferred way to go?
>
>
>
> Best regards,
>
> Sebastian
>
>
>
> *Von:* Stian Thorgersen <sthorger at redhat.com>
> *Gesendet:* Montag, 3. September 2018 12:21
> *An:* L?sch, Sebastian <Sebastian.Loesch at governikus.de>
> *Cc:* keycloak-dev <keycloak-dev at lists.jboss.org>
> *Betreff:* Re: [keycloak-dev] SCIM v2 support
>
>
>
> Adding additional attributes to user entity is probably not the way to go.
> Rather, it would be better for backwards compatibility to simply use
> generic key/value attributes which the user entity already has.
>
>
>
> Implementing the SCIM endpoints is probably pretty straightforward. Most
> of the work will probably be down to testing and documentation.
>
> On Wed, 29 Aug 2018 at 13:28, L?sch, Sebastian <
> Sebastian.Loesch at governikus.de> wrote:
>
> Hello,
>
>
>
> in a customer project we use keycloak and need a SCIM (System for
> Cross-domain Identity Management) API.
>
> Currently we write a wrapper API and a custom endpoint providing the SCIM
> functionality. We wrote a extension of the UserEntity, UserModel and an
> extension of the JpaUserProvider.
>
> This strategy seems not ideal and the nicest way is to add this extensions
> to Keycloak. This is already suggested in
> https://issues.jboss.org/browse/KEYCLOAK-2537
>
> Is anybody out there who can guide me, what coding would be necessary to
> contribute the SCIM functionality?
>
>
>
> Currently I think we have to:
>
> -          extend the UserEntity with all SCIM attributes. This will result
> in additional tables/entities for complex attributes e.g. Address, Name,
> Email
>
> -          extend the UserModel to povide the additional attributes
>
> -          implement the new SCIM endpoint /Users
>
> -          make the additional attributes available via Admin REST API
> /users
>
> -          extend views to be able to edit SCIM user attributes using the
> web ui
>
> -          ?
>
> -          All the above again for the Groups endpoint?
>
>
>
> This also seem to be major changes. To big for one Pull Request. How do you
> like to handle this?
>
>
>
> Best regards,
>
> Sebastian
>
>
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>

From tushardhole at hotmail.com  Tue Sep  4 08:29:17 2018
From: tushardhole at hotmail.com (tushar dhole)
Date: Tue, 4 Sep 2018 12:29:17 +0000
Subject: [keycloak-dev] Keycloak large token issue
Message-ID: <HK0PR01MB2113A802213E4383ECD8F399AC030@HK0PR01MB2113.apcprd01.prod.exchangelabs.com>

Hello Community,

There is a limitation with keycloak to support large number of realms. Following is the existing jira issue related to same,


https://issues.jboss.org/browse/KEYCLOAK-1268


I was wondering if we can solve this following approach,


  1.  Make the token for users under "master" realm not return all realm info.
  2.  We can just allow a token from "realm": "master" any for cross realm authorization
  3.  For all other non master realm the token will be same as that of today


If this approach is feasible/doable, then I can dig into the code and try to implement this.
But would first like to know if this is really a feasible/doable one.

Thanks a lot,
Tushar Dhole
<https://issues.jboss.org/browse/KEYCLOAK-1268>
[KEYCLOAK-1268] Token for admin becomes to large with many ...<https://issues.jboss.org/browse/KEYCLOAK-1268>
issues.jboss.org
I have a big problem here because of bearer token size. I'm using keycloak within a SaaS application, so I need create alot of realms.






From sthorger at redhat.com  Tue Sep  4 09:29:20 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Tue, 4 Sep 2018 15:29:20 +0200
Subject: [keycloak-dev] Permission for client scopes
Message-ID: <CAJgngAcC132GwvCuA0=Comy1cmrKE1z7bCOsYyMH3_PYpnLoeg@mail.gmail.com>

As scopes are often used for permissions in the applications themselves it
would be useful to have a mechanism to grant a user access to a scope.

For example if you have the scopes "photos:view" and "photos:edit" you
would like only users that are permitted to use the photos application to
be able to get those scopes in the token.

One simple way of doing this would be to have a optional required role
associated with a client scope. Then we can simply apply the client scopes
for which the user has the required role.

From sthorger at redhat.com  Tue Sep  4 15:10:52 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Tue, 4 Sep 2018 21:10:52 +0200
Subject: [keycloak-dev] Keycloak large token issue
In-Reply-To: <HK0PR01MB2113A802213E4383ECD8F399AC030@HK0PR01MB2113.apcprd01.prod.exchangelabs.com>
References: <HK0PR01MB2113A802213E4383ECD8F399AC030@HK0PR01MB2113.apcprd01.prod.exchangelabs.com>
Message-ID: <CAJgngAe7shRc4Qzba_XTCOHPaS8+P+t0q6jMiT_HXMNinzvVKw@mail.gmail.com>

This should already be resolved and the tokens issues to the admin console
should not contain all roles anymore. Have you tried this with the latest
release?

On Tue, 4 Sep 2018 at 14:30, tushar dhole <tushardhole at hotmail.com> wrote:

> Hello Community,
>
> There is a limitation with keycloak to support large number of realms.
> Following is the existing jira issue related to same,
>
>
> https://issues.jboss.org/browse/KEYCLOAK-1268
>
>
> I was wondering if we can solve this following approach,
>
>
>   1.  Make the token for users under "master" realm not return all realm
> info.
>   2.  We can just allow a token from "realm": "master" any for cross realm
> authorization
>   3.  For all other non master realm the token will be same as that of
> today
>
>
> If this approach is feasible/doable, then I can dig into the code and try
> to implement this.
> But would first like to know if this is really a feasible/doable one.
>
> Thanks a lot,
> Tushar Dhole
> <https://issues.jboss.org/browse/KEYCLOAK-1268>
> [KEYCLOAK-1268] Token for admin becomes to large with many ...<
> https://issues.jboss.org/browse/KEYCLOAK-1268>
> issues.jboss.org
> I have a big problem here because of bearer token size. I'm using keycloak
> within a SaaS application, so I need create alot of realms.
>
>
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From tushardhole at hotmail.com  Tue Sep  4 22:56:58 2018
From: tushardhole at hotmail.com (tushar dhole)
Date: Wed, 5 Sep 2018 02:56:58 +0000
Subject: [keycloak-dev] Keycloak large token issue
In-Reply-To: <CAJgngAe7shRc4Qzba_XTCOHPaS8+P+t0q6jMiT_HXMNinzvVKw@mail.gmail.com>
References: <HK0PR01MB2113A802213E4383ECD8F399AC030@HK0PR01MB2113.apcprd01.prod.exchangelabs.com>,
	<CAJgngAe7shRc4Qzba_XTCOHPaS8+P+t0q6jMiT_HXMNinzvVKw@mail.gmail.com>
Message-ID: <HK0PR01MB21137D98EA16F61CA0ED6EEFAC020@HK0PR01MB2113.apcprd01.prod.exchangelabs.com>

Oh that's great .. Thanks.. I will try latest...

Get Outlook for Android<https://aka.ms/ghei36>



From: Stian Thorgersen
Sent: Wednesday, 5 September, 12:41 am
Subject: Re: [keycloak-dev] Keycloak large token issue
To: tushardhole at hotmail.com
Cc: keycloak-dev


This should already be resolved and the tokens issues to the admin console should not contain all roles anymore. Have you tried this with the latest release?

On Tue, 4 Sep 2018 at 14:30, tushar dhole <tushardhole at hotmail.com<mailto:tushardhole at hotmail.com>> wrote:
Hello Community,

There is a limitation with keycloak to support large number of realms. Following is the existing jira issue related to same,


https://issues.jboss.org/browse/KEYCLOAK-1268


I was wondering if we can solve this following approach,


  1.  Make the token for users under "master" realm not return all realm info.
  2.  We can just allow a token from "realm": "master" any for cross realm authorization
  3.  For all other non master realm the token will be same as that of today


If this approach is feasible/doable, then I can dig into the code and try to implement this.
But would first like to know if this is really a feasible/doable one.

Thanks a lot,
Tushar Dhole
<https://issues.jboss.org/browse/KEYCLOAK-1268>
[KEYCLOAK-1268] Token for admin becomes to large with many ...<https://issues.jboss.org/browse/KEYCLOAK-1268>
issues.jboss.org<http://issues.jboss.org>
I have a big problem here because of bearer token size. I'm using keycloak within a SaaS application, so I need create alot of realms.





_______________________________________________
keycloak-dev mailing list
keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-dev



From pasik at iki.fi  Wed Sep  5 10:03:03 2018
From: pasik at iki.fi (Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?=)
Date: Wed, 5 Sep 2018 17:03:03 +0300
Subject: [keycloak-dev] Permission for client scopes
In-Reply-To: <CAJgngAcC132GwvCuA0=Comy1cmrKE1z7bCOsYyMH3_PYpnLoeg@mail.gmail.com>
References: <CAJgngAcC132GwvCuA0=Comy1cmrKE1z7bCOsYyMH3_PYpnLoeg@mail.gmail.com>
Message-ID: <20180905140303.GB18222@reaktio.net>

On Tue, Sep 04, 2018 at 03:29:20PM +0200, Stian Thorgersen wrote:
> As scopes are often used for permissions in the applications themselves it
> would be useful to have a mechanism to grant a user access to a scope.
> 
> For example if you have the scopes "photos:view" and "photos:edit" you
> would like only users that are permitted to use the photos application to
> be able to get those scopes in the token.
> 
> One simple way of doing this would be to have a optional required role
> associated with a client scope. Then we can simply apply the client scopes
> for which the user has the required role.
>

+1

Something like this is definitely needed and useful in Keycloak.

I guess this is: https://issues.jboss.org/browse/KEYCLOAK-8175


-- Pasi


From mposolda at redhat.com  Thu Sep  6 16:28:13 2018
From: mposolda at redhat.com (Marek Posolda)
Date: Thu, 6 Sep 2018 22:28:13 +0200
Subject: [keycloak-dev] Audience support
Message-ID: <2deecc7b-f8b1-c2c0-a958-abc3bd2eed43@redhat.com>

Hi,

I have some concerns about the audience support. The prototype done few 
months ago, which I have in my branch [1] is based on client scopes. I 
showed it on the weekly demo some time ago and changed few things based 
on the feedback. Just to clarify, the PR:
- Adds the AudienceProtocolMapper, which is able to attach single 
client_id of the specified client (typically will be bearer-only client)

- Adds the easy way to generate "audience client scope" for specified 
"service" client (usually bearer-only client). This clientScope contains:
-- the audienceProtocolMapper of bearer-only described above
-- the scope role mappings of all the client roles of that bearer-only 
client.
The "frontend" clients can then use this clientScope, so that audience + 
roleMappings will be both available in the accessToken issued for the 
frontend-client.

Now there is this specs [2], which Pedro pointed. Short summary about 
this specs:

- It is still in draft state for few years and there are some TODO in 
it. So hard to say how much to rely on it...

- It specifically talks about "scope" parameter and mentions, that it is 
often used by various parties for "what the token can be used for" 
(permissions/claims associated with the token) and "where to use that 
token" . But it mentions that "where" use-case should be ideally handled 
by something else.

- It introduces "resource" parameter, which MUST be an absolute URI per 
this specification and it should point to the service/resource where the 
access token will be used. When authorization server is creating token, 
it may apply audiences, encrypt token by various algorithms based on the 
requested "resource" etc. For the audience, they specify that:

"The authorization server may use the exact 'resource' value as the 
audience or it may map from that value to a more general URI or abstract 
identifier for the resource server."

- The "resource" parameter is used in the token request if possible. For 
the grants where it's not possible (implicit, resource owner password 
login, client credentials) it is used in the initial authentication 
request. So for the classic "authorization_code" grant it can be used in 
code-to-token or refresh-token requests. This is great and allows some 
cool flexibility (unlike "scope" where the whole SSO login including all 
the redirects is needed). But it requires some support on adapters.


Now I am thinking about 2 possible ways to continue:

1) Finish my current work around audience and for now, let it be driven 
through the "scope" parameter. Then later add the support for the 
"resource" parameter and the specs above.
2) Skip what I currently have and start something different based on the 
"resource" parameter specs.

ATM I am more keen to option 1, so that we can have some audience 
support in master ASAP. Also because the specs is still draft and I 
don't know how much we can rely on the "resource" parameter being 
adopted by 3rd party applications (for example Openshift). When we later 
have the option to drive audiences through both "scope" and "resource" 
params, it will be good thing IMO.

The "resource" related support will require changes on the adapter side 
as it is new parameter to be supported in code-to-token request and 
refresh-token request. On our adapter's side, it will be good to have 
some changes to allow "cache" of more access tokens for various 
resources/audiences, not just one. On server-side, I was thinking about 
various possibilities, and it still looks good to represent "resource" 
support through client scopes. So optional clientScopes can be requested 
either through "scope" parameter (default case) or through "resource" 
parameter (probably disabled by default) or through both.

Thoughts?


[1] https://github.com/mposolda/keycloak/tree/audience
[2] https://tools.ietf.org/html/draft-ietf-oauth-resource-indicators-00

Marek


From sthorger at redhat.com  Fri Sep  7 02:20:45 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Fri, 7 Sep 2018 08:20:45 +0200
Subject: [keycloak-dev] Keycloak 4.4.0.Final is out
Message-ID: <CAJgngAcUNAHVjiZh9gUrck0PsbzY_wjhSnqcQyVYG755-Qj3Hg@mail.gmail.com>

https://www.keycloak.org/docs/latest/release_notes/index.html

From ian at ianduffy.ie  Fri Sep  7 05:31:58 2018
From: ian at ianduffy.ie (Ian Duffy)
Date: Fri, 7 Sep 2018 10:31:58 +0100
Subject: [keycloak-dev] Keycloak 4.4.0.Final is out
Message-ID: <CAEJ3w4VUHP99Ru7xxO4y3arGtrORObfxxU-56jZtS9X7HVHyqQ@mail.gmail.com>

Hi all,

Glad to see the Keycloak 4.4.0 release, the image seems to be missing from
Docker Hub.

https://hub.docker.com/r/jboss/keycloak/tags/

Is there anything that can be done to fix this?

Thanks,
Ian.

From it.vidhyadharan at gmail.com  Sun Sep  9 13:11:16 2018
From: it.vidhyadharan at gmail.com (vidhyadharan D)
Date: Sun, 9 Sep 2018 22:41:16 +0530
Subject: [keycloak-dev] Regarding
	https://issues.jboss.org/browse/KEYCLOAK-8162
Message-ID: <CAJuVHDytM+FbAxJr88BwWN1Q7rzOcppyv+mtBNs8-77vX5emuQ@mail.gmail.com>

Hi Experts,

I have been working on keycloak html email. I am in need to embed  logo  to
the emails i.e. from the *themes/email/resources/img/logo.png *

 In the login module there is a way to locate image / favicon via
${url.resourcesPath}

However in email module we dont have access to locate the resources..

I have achieved by adding custom email template provider. If possible
please add these into email module because it is useful for all .

or let me know i can provide PR.

https://issues.jboss.org/browse/KEYCLOAK-8162


Thanks,
vidhya

From sthorger at redhat.com  Mon Sep 10 02:57:28 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 10 Sep 2018 08:57:28 +0200
Subject: [keycloak-dev] Keycloak 4.4.0.Final is out
In-Reply-To: <CAEJ3w4VUHP99Ru7xxO4y3arGtrORObfxxU-56jZtS9X7HVHyqQ@mail.gmail.com>
References: <CAEJ3w4VUHP99Ru7xxO4y3arGtrORObfxxU-56jZtS9X7HVHyqQ@mail.gmail.com>
Message-ID: <CAJgngAf_ndgxpC912r16Ru9=76Zp=TXfWiZ+ms3kRV3SFDu89w@mail.gmail.com>

Build on its way now.

On Fri, 7 Sep 2018 at 11:32, Ian Duffy <ian at ianduffy.ie> wrote:

> Hi all,
>
> Glad to see the Keycloak 4.4.0 release, the image seems to be missing from
> Docker Hub.
>
> https://hub.docker.com/r/jboss/keycloak/tags/
>
> Is there anything that can be done to fix this?
>
> Thanks,
> Ian.
>

From sthorger at redhat.com  Mon Sep 10 03:15:59 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 10 Sep 2018 09:15:59 +0200
Subject: [keycloak-dev] Regarding
	https://issues.jboss.org/browse/KEYCLOAK-8162
In-Reply-To: <CAJuVHDytM+FbAxJr88BwWN1Q7rzOcppyv+mtBNs8-77vX5emuQ@mail.gmail.com>
References: <CAJuVHDytM+FbAxJr88BwWN1Q7rzOcppyv+mtBNs8-77vX5emuQ@mail.gmail.com>
Message-ID: <CAJgngAe=O9c5SCRdQB5pxs0UhDvq5ih+PJC-1CXkx1jzpfi3Yg@mail.gmail.com>

For emails shouldn't images actually be encoded into the email itself
rather than linked to?

On Sun, 9 Sep 2018 at 19:18, vidhyadharan D <it.vidhyadharan at gmail.com>
wrote:

> Hi Experts,
>
> I have been working on keycloak html email. I am in need to embed  logo  to
> the emails i.e. from the *themes/email/resources/img/logo.png *
>
>  In the login module there is a way to locate image / favicon via
> ${url.resourcesPath}
>
> However in email module we dont have access to locate the resources..
>
> I have achieved by adding custom email template provider. If possible
> please add these into email module because it is useful for all .
>
> or let me know i can provide PR.
>
> https://issues.jboss.org/browse/KEYCLOAK-8162
>
>
> Thanks,
> vidhya
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From sthorger at redhat.com  Mon Sep 10 03:21:00 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 10 Sep 2018 09:21:00 +0200
Subject: [keycloak-dev] Audience support
In-Reply-To: <2deecc7b-f8b1-c2c0-a958-abc3bd2eed43@redhat.com>
References: <2deecc7b-f8b1-c2c0-a958-abc3bd2eed43@redhat.com>
Message-ID: <CAJgngAftF0xDDdBpjX+H7p5Kxbj91jELYmWQ5PZbk2XNKz3-iQ@mail.gmail.com>

As aud field is widely used today that is what we need now. I haven't seen
anyone ask for [2] so I think we can safely ignore that spec for now.

I would finish what you have with aud support for now, then we can address
[2] in the future if needed.

On Thu, 6 Sep 2018 at 23:37, Marek Posolda <mposolda at redhat.com> wrote:

> Hi,
>
> I have some concerns about the audience support. The prototype done few
> months ago, which I have in my branch [1] is based on client scopes. I
> showed it on the weekly demo some time ago and changed few things based
> on the feedback. Just to clarify, the PR:
> - Adds the AudienceProtocolMapper, which is able to attach single
> client_id of the specified client (typically will be bearer-only client)
>
> - Adds the easy way to generate "audience client scope" for specified
> "service" client (usually bearer-only client). This clientScope contains:
> -- the audienceProtocolMapper of bearer-only described above
> -- the scope role mappings of all the client roles of that bearer-only
> client.
> The "frontend" clients can then use this clientScope, so that audience +
> roleMappings will be both available in the accessToken issued for the
> frontend-client.
>
> Now there is this specs [2], which Pedro pointed. Short summary about
> this specs:
>
> - It is still in draft state for few years and there are some TODO in
> it. So hard to say how much to rely on it...
>
> - It specifically talks about "scope" parameter and mentions, that it is
> often used by various parties for "what the token can be used for"
> (permissions/claims associated with the token) and "where to use that
> token" . But it mentions that "where" use-case should be ideally handled
> by something else.
>
> - It introduces "resource" parameter, which MUST be an absolute URI per
> this specification and it should point to the service/resource where the
> access token will be used. When authorization server is creating token,
> it may apply audiences, encrypt token by various algorithms based on the
> requested "resource" etc. For the audience, they specify that:
>
> "The authorization server may use the exact 'resource' value as the
> audience or it may map from that value to a more general URI or abstract
> identifier for the resource server."
>
> - The "resource" parameter is used in the token request if possible. For
> the grants where it's not possible (implicit, resource owner password
> login, client credentials) it is used in the initial authentication
> request. So for the classic "authorization_code" grant it can be used in
> code-to-token or refresh-token requests. This is great and allows some
> cool flexibility (unlike "scope" where the whole SSO login including all
> the redirects is needed). But it requires some support on adapters.
>
>
> Now I am thinking about 2 possible ways to continue:
>
> 1) Finish my current work around audience and for now, let it be driven
> through the "scope" parameter. Then later add the support for the
> "resource" parameter and the specs above.
> 2) Skip what I currently have and start something different based on the
> "resource" parameter specs.
>
> ATM I am more keen to option 1, so that we can have some audience
> support in master ASAP. Also because the specs is still draft and I
> don't know how much we can rely on the "resource" parameter being
> adopted by 3rd party applications (for example Openshift). When we later
> have the option to drive audiences through both "scope" and "resource"
> params, it will be good thing IMO.
>
> The "resource" related support will require changes on the adapter side
> as it is new parameter to be supported in code-to-token request and
> refresh-token request. On our adapter's side, it will be good to have
> some changes to allow "cache" of more access tokens for various
> resources/audiences, not just one. On server-side, I was thinking about
> various possibilities, and it still looks good to represent "resource"
> support through client scopes. So optional clientScopes can be requested
> either through "scope" parameter (default case) or through "resource"
> parameter (probably disabled by default) or through both.
>
> Thoughts?
>
>
> [1] https://github.com/mposolda/keycloak/tree/audience
> [2] https://tools.ietf.org/html/draft-ietf-oauth-resource-indicators-00
>
> Marek
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From sthorger at redhat.com  Mon Sep 10 03:22:07 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 10 Sep 2018 09:22:07 +0200
Subject: [keycloak-dev] Possible feature: role attributes
In-Reply-To: <365a8b409e5248908b2eb312a6d202bc@bosch-si.com>
References: <10b624a57cea40a1bcc9bd77b63b398a@bosch-si.com>
	<CAJgngAfPK97Bsy3V5fq4ZU5FAYkzCDL1spGwtQ+MvHDgfzsaxw@mail.gmail.com>
	<365a8b409e5248908b2eb312a6d202bc@bosch-si.com>
Message-ID: <CAJgngAe+ygTaCai=BNVpmqEguD4sAekmE2qM2nYzisZa3EQ0iw@mail.gmail.com>

Sounds good. I can't think of anything that you've missed from the list.

On Fri, 31 Aug 2018 at 15:46, Schuster Sebastian (INST/ESY1) <
Sebastian.Schuster at bosch-si.com> wrote:

> All right. I would like to create a prototype for this. I would take
> inspiration from the way custom group attributes are currently implemented.
>
> I guess changes would be necessary in the following areas:
>
> ?         DB schema
>
> ?         Persistence layer
>
> ?         Caching layer
>
> ?         CRUD API
>
> ?         Admin console
>
> ?         Admin CLI
>
> ?         Java client
>
> ?         Admin events
>
> Anything I missed?
>
>
>
> Thanks and best regards,
>
> Sebastian
>
>
>
> Mit freundlichen Gr??en / Best regards
>
>
> *Dr.-Ing. Sebastian Schuster *
> Engineering and Support (INST/ESY1)
> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> GERMANY | www.bosch-si.com
> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
> Sebastian.Schuster at bosch-si.com
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr.
> Stefan Ferber, Michael Hahn
>
>
>
> *From:* Stian Thorgersen <sthorger at redhat.com>
> *Sent:* Montag, 27. August 2018 13:49
> *To:* Schuster Sebastian (INST/ESY1) <Sebastian.Schuster at bosch-si.com>
> *Cc:* keycloak-dev <keycloak-dev at lists.jboss.org>
> *Subject:* Re: [keycloak-dev] Possible feature: role attributes
>
>
>
> I don't think we need to consider adding role attributes to the token.
> That would very quickly bloat tokens.
>
>
>
> I would like to see a bit more general use of role attributes as part of
> incorporating such a feature. Otherwise it would end up being a rather
> hidden feature. Some ideas I have in mind:
>
>
>
> * Ability to do crud of role attributes in admin console
>
> * Ability to query for roles based on attributes
>
>
>
> For future work it would be great to have attributes on everything. That
> would allow us to do something like OpenShift `oc` does. Where you're able
> to search and delete everything based on attributes. One nice use-case here
> is that you can tag all clients, roles, etc.. that belong to a deployment
> (a group of apps and services) and be able to view everything that is
> related to the deployment in Keycloak.
>
>
>
> On Mon, 27 Aug 2018 at 13:32, Schuster Sebastian (INST/ESY1) <
> Sebastian.Schuster at bosch-si.com> wrote:
>
> Hi everybody,
>
> We have a use case where we would like to store additional
> meta-information for roles. This come from our IAM-requirements, that say
> there is a single responsible person for a role or that roles give access
> to data with different classifications. One way to store this kind of
> information would be to introduce role attributes to client and realm
> roles, basically similar to user or group attributes.
>
> For us, it would be sufficient to have this information purely as
> metadata, i.e. we would only read it through the audit log to inform the
> responsible person about role assignments if a role with a certain
> classification is assigned. In contrast to that, you can add group und user
> attributes to a token using user attribute mappers and the client
> application can extract this information from the token and act on it.
>
> WDYT? Does anybody else have similar requirements? Would you need role
> custom attributes also in the token? I can imagine that it gets kind of
> difficult to identify where attributes come from, once there are user,
> group, and role attributes, possibly with inheritance/composition.
>
> Best regards,
> Sebastian
>
> Mit freundlichen Gr??en / Best regards
>
> Dr.-Ing. Sebastian Schuster
>
> Engineering and Support (INST/ESY1)
> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> GERMANY | www.bosch-si.com<http://www.bosch-si.com>
> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
> Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr.
> Stefan Ferber, Michael Hahn
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>

From mposolda at redhat.com  Mon Sep 10 03:37:05 2018
From: mposolda at redhat.com (Marek Posolda)
Date: Mon, 10 Sep 2018 09:37:05 +0200
Subject: [keycloak-dev] Audience support
In-Reply-To: <CAJgngAftF0xDDdBpjX+H7p5Kxbj91jELYmWQ5PZbk2XNKz3-iQ@mail.gmail.com>
References: <2deecc7b-f8b1-c2c0-a958-abc3bd2eed43@redhat.com>
	<CAJgngAftF0xDDdBpjX+H7p5Kxbj91jELYmWQ5PZbk2XNKz3-iQ@mail.gmail.com>
Message-ID: <eb23a8cc-6b2c-fe92-5657-ea5ba41d7ccb@redhat.com>

Cool, should have audience PR ready in next few days then. Will create 
"future" version JIRA for the "Resource" specs.

Marek

On 10/09/18 09:21, Stian Thorgersen wrote:
> As aud field is widely used today that is what we need now. I haven't 
> seen anyone ask for [2] so I think we can safely ignore that spec for 
> now.
>
> I would finish what you have with aud support for now, then we can 
> address [2] in the future if needed.
>
> On Thu, 6 Sep 2018 at 23:37, Marek Posolda <mposolda at redhat.com 
> <mailto:mposolda at redhat.com>> wrote:
>
>     Hi,
>
>     I have some concerns about the audience support. The prototype
>     done few
>     months ago, which I have in my branch [1] is based on client
>     scopes. I
>     showed it on the weekly demo some time ago and changed few things
>     based
>     on the feedback. Just to clarify, the PR:
>     - Adds the AudienceProtocolMapper, which is able to attach single
>     client_id of the specified client (typically will be bearer-only
>     client)
>
>     - Adds the easy way to generate "audience client scope" for specified
>     "service" client (usually bearer-only client). This clientScope
>     contains:
>     -- the audienceProtocolMapper of bearer-only described above
>     -- the scope role mappings of all the client roles of that
>     bearer-only
>     client.
>     The "frontend" clients can then use this clientScope, so that
>     audience +
>     roleMappings will be both available in the accessToken issued for the
>     frontend-client.
>
>     Now there is this specs [2], which Pedro pointed. Short summary about
>     this specs:
>
>     - It is still in draft state for few years and there are some TODO in
>     it. So hard to say how much to rely on it...
>
>     - It specifically talks about "scope" parameter and mentions, that
>     it is
>     often used by various parties for "what the token can be used for"
>     (permissions/claims associated with the token) and "where to use that
>     token" . But it mentions that "where" use-case should be ideally
>     handled
>     by something else.
>
>     - It introduces "resource" parameter, which MUST be an absolute
>     URI per
>     this specification and it should point to the service/resource
>     where the
>     access token will be used. When authorization server is creating
>     token,
>     it may apply audiences, encrypt token by various algorithms based
>     on the
>     requested "resource" etc. For the audience, they specify that:
>
>     "The authorization server may use the exact 'resource' value as the
>     audience or it may map from that value to a more general URI or
>     abstract
>     identifier for the resource server."
>
>     - The "resource" parameter is used in the token request if
>     possible. For
>     the grants where it's not possible (implicit, resource owner password
>     login, client credentials) it is used in the initial authentication
>     request. So for the classic "authorization_code" grant it can be
>     used in
>     code-to-token or refresh-token requests. This is great and allows
>     some
>     cool flexibility (unlike "scope" where the whole SSO login
>     including all
>     the redirects is needed). But it requires some support on adapters.
>
>
>     Now I am thinking about 2 possible ways to continue:
>
>     1) Finish my current work around audience and for now, let it be
>     driven
>     through the "scope" parameter. Then later add the support for the
>     "resource" parameter and the specs above.
>     2) Skip what I currently have and start something different based
>     on the
>     "resource" parameter specs.
>
>     ATM I am more keen to option 1, so that we can have some audience
>     support in master ASAP. Also because the specs is still draft and I
>     don't know how much we can rely on the "resource" parameter being
>     adopted by 3rd party applications (for example Openshift). When we
>     later
>     have the option to drive audiences through both "scope" and
>     "resource"
>     params, it will be good thing IMO.
>
>     The "resource" related support will require changes on the adapter
>     side
>     as it is new parameter to be supported in code-to-token request and
>     refresh-token request. On our adapter's side, it will be good to have
>     some changes to allow "cache" of more access tokens for various
>     resources/audiences, not just one. On server-side, I was thinking
>     about
>     various possibilities, and it still looks good to represent
>     "resource"
>     support through client scopes. So optional clientScopes can be
>     requested
>     either through "scope" parameter (default case) or through "resource"
>     parameter (probably disabled by default) or through both.
>
>     Thoughts?
>
>
>     [1] https://github.com/mposolda/keycloak/tree/audience
>     [2]
>     https://tools.ietf.org/html/draft-ietf-oauth-resource-indicators-00
>
>     Marek
>
>     _______________________________________________
>     keycloak-dev mailing list
>     keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/keycloak-dev
>


From psilva at redhat.com  Mon Sep 10 07:56:44 2018
From: psilva at redhat.com (Pedro Igor Silva)
Date: Mon, 10 Sep 2018 08:56:44 -0300
Subject: [keycloak-dev] Audience support
In-Reply-To: <2deecc7b-f8b1-c2c0-a958-abc3bd2eed43@redhat.com>
References: <2deecc7b-f8b1-c2c0-a958-abc3bd2eed43@redhat.com>
Message-ID: <CAJrcDBeMP_y4HeTQE9hLq9-eW8tKGXGytQJMGoeDQpKKKftrTA@mail.gmail.com>

Hi Marek,

I think "resource" will be a good addition but we are already capable of
choosing/processing audiences using scopes. So yeah, IMO, we can go for #1.

On Thu, Sep 6, 2018 at 5:28 PM, Marek Posolda <mposolda at redhat.com> wrote:

> Hi,
>
> I have some concerns about the audience support. The prototype done few
> months ago, which I have in my branch [1] is based on client scopes. I
> showed it on the weekly demo some time ago and changed few things based
> on the feedback. Just to clarify, the PR:
> - Adds the AudienceProtocolMapper, which is able to attach single
> client_id of the specified client (typically will be bearer-only client)
>
> - Adds the easy way to generate "audience client scope" for specified
> "service" client (usually bearer-only client). This clientScope contains:
> -- the audienceProtocolMapper of bearer-only described above
> -- the scope role mappings of all the client roles of that bearer-only
> client.
> The "frontend" clients can then use this clientScope, so that audience +
> roleMappings will be both available in the accessToken issued for the
> frontend-client.
>
> Now there is this specs [2], which Pedro pointed. Short summary about
> this specs:
>
> - It is still in draft state for few years and there are some TODO in
> it. So hard to say how much to rely on it...
>
> - It specifically talks about "scope" parameter and mentions, that it is
> often used by various parties for "what the token can be used for"
> (permissions/claims associated with the token) and "where to use that
> token" . But it mentions that "where" use-case should be ideally handled
> by something else.
>
> - It introduces "resource" parameter, which MUST be an absolute URI per
> this specification and it should point to the service/resource where the
> access token will be used. When authorization server is creating token,
> it may apply audiences, encrypt token by various algorithms based on the
> requested "resource" etc. For the audience, they specify that:
>
> "The authorization server may use the exact 'resource' value as the
> audience or it may map from that value to a more general URI or abstract
> identifier for the resource server."
>
> - The "resource" parameter is used in the token request if possible. For
> the grants where it's not possible (implicit, resource owner password
> login, client credentials) it is used in the initial authentication
> request. So for the classic "authorization_code" grant it can be used in
> code-to-token or refresh-token requests. This is great and allows some
> cool flexibility (unlike "scope" where the whole SSO login including all
> the redirects is needed). But it requires some support on adapters.
>
>
> Now I am thinking about 2 possible ways to continue:
>
> 1) Finish my current work around audience and for now, let it be driven
> through the "scope" parameter. Then later add the support for the
> "resource" parameter and the specs above.
> 2) Skip what I currently have and start something different based on the
> "resource" parameter specs.
>
> ATM I am more keen to option 1, so that we can have some audience
> support in master ASAP. Also because the specs is still draft and I
> don't know how much we can rely on the "resource" parameter being
> adopted by 3rd party applications (for example Openshift). When we later
> have the option to drive audiences through both "scope" and "resource"
> params, it will be good thing IMO.
>
> The "resource" related support will require changes on the adapter side
> as it is new parameter to be supported in code-to-token request and
> refresh-token request. On our adapter's side, it will be good to have
> some changes to allow "cache" of more access tokens for various
> resources/audiences, not just one. On server-side, I was thinking about
> various possibilities, and it still looks good to represent "resource"
> support through client scopes. So optional clientScopes can be requested
> either through "scope" parameter (default case) or through "resource"
> parameter (probably disabled by default) or through both.
>
> Thoughts?
>
>
> [1] https://github.com/mposolda/keycloak/tree/audience
> [2] https://tools.ietf.org/html/draft-ietf-oauth-resource-indicators-00
>
> Marek
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From Chris.Phillips at canarie.ca  Mon Sep 10 10:55:40 2018
From: Chris.Phillips at canarie.ca (Chris Phillips)
Date: Mon, 10 Sep 2018 14:55:40 +0000
Subject: [keycloak-dev] Can KeyCloak support Multi-lateral SAML federation?
Message-ID: <F365057E-04C2-4A26-AFAA-F9FEE43A51FE@canarie.ca>

Hi. 
I sent this to the Users list and have had zero response.  Re-sending here on the dev list hoping to hear feedback and thoughts from Keycloak Devs on my questions around KeyCloak's ability to support multi-lateral federation and if it is on the roadmap.

Thanks and look forward to thoughts and comments..

Chris.


?On 2018-08-30, 4:06 PM, "keycloak-user-bounces at lists.jboss.org on behalf of Chris Phillips" <keycloak-user-bounces at lists.jboss.org on behalf of Chris.Phillips at canarie.ca> wrote:

    Hi.
    I?m going through assessing KeyCloak as being able to be an Identity Provider in a multi-lateral SAML federation context and am seeking insight from the users and devs involved in KeyCloak.
    
    For an IdP to be considered interoperable in a multi-lateral SAML trust federation context,  IdPs need to be able to do a base set of functions. These are some of the critical (but not only) ones:
    
      *   Retrieve, with a configurable frequency (usually hourly), an online metadata aggregate
      *   validate the signature on the aggregate
      *   when signature validity is verified, load all the entities (Identity Providers/Service Providers) to be trusted or used in trust decisions in the Identity Provider.
    
    I have not seen this capability in KeyCloak 4.3.0.Final (docker) but could be missing something.
    
    Is anyone using KeyCloak in this manner or are there plans for this functionality on KeyCloak?s technical roadmap?
    
    Some additional items to decorate my ask for information..
    
    To give an idea of scale, the aggregates I want to work with have ~4500 entities with 2800 IdPs and 2100 SPs and need to  be refreshed hourly.
    
    The list of items important for interoperability can be seen here with the ones I called out above appearing in section 2.2.1:
    https://kantarainitiative.github.io/SAMLprofiles/fedinterop.html
    
    
    I?ve searched the keycloak-users list a bit and came across the reference to EntitiesDescriptor which lead me to this issue and code update in KeyCloak: https://issues.jboss.org/browse/KEYCLOAK-4399 which leads me to think that the support for reading in aggregates is not possible and maybe engineered out of the product itself.  Am I right in thinking that?
    
    
    Thoughts and insights welcome..
    
    Chris.
    ___________________________________________________________________________________________
    Chris Phillips
    Technical Architect, Canadian Access Federation, CANARIE| chris.phillips at canarie.ca<mailto:chris.phillips at canarie.ca>  |GPG: 0x7F6245580380811D
    
    _______________________________________________
    keycloak-user mailing list
    keycloak-user at lists.jboss.org
    https://lists.jboss.org/mailman/listinfo/keycloak-user



From ssilvert at redhat.com  Mon Sep 10 19:05:26 2018
From: ssilvert at redhat.com (Stan Silvert)
Date: Mon, 10 Sep 2018 19:05:26 -0400
Subject: [keycloak-dev] Avoiding XSS in Keycloak/Freemarker
Message-ID: <ad8cf6d1-de36-94dc-8013-7b1876bde4fc@redhat.com>

*Executive summary: *Any time you use ?no_esc, make sure you also use 
kcSanitize().

Most of you on the team are already aware of this, but for completeness 
I'll post to the community dev list.

Some time ago, we switched our default Freemarker settings so that 
everything is escaped by default.? However, this means that some things 
need to be explicitly "un-escaped".? This includes any time we allow 
HTML in an admin console field or in a resource bundle (*.properties file).

The way this is done in Freemarker is to use the ?no_esc directive.? For 
example, login.ftl has

<span class="kc-feedback-text">${message.summary?no_esc}</span>

This is fine, but what if, somehow an attacker was able to insert some 
HTML into message.summary?? It's unlikely because it requires a high 
level of access.? But it's possible.

We now protect against this using a sanitizer for unescaped content.? 
 From now on, you should call the Keycloak sanitizer any time you are 
using ?no_esc.? The sanitizer removes unsafe HTML using an algorithm 
similar to the one used on eBay.? So this example becomes:

<span class="kc-feedback-text">${kcSanitize(message.summary)?no_esc}</span>



From it.vidhyadharan at gmail.com  Tue Sep 11 04:07:14 2018
From: it.vidhyadharan at gmail.com (vidhyadharan D)
Date: Tue, 11 Sep 2018 13:37:14 +0530
Subject: [keycloak-dev] Regarding
	https://issues.jboss.org/browse/KEYCLOAK-8162
In-Reply-To: <CAJgngAe=O9c5SCRdQB5pxs0UhDvq5ih+PJC-1CXkx1jzpfi3Yg@mail.gmail.com>
References: <CAJuVHDytM+FbAxJr88BwWN1Q7rzOcppyv+mtBNs8-77vX5emuQ@mail.gmail.com>
	<CAJgngAe=O9c5SCRdQB5pxs0UhDvq5ih+PJC-1CXkx1jzpfi3Yg@mail.gmail.com>
Message-ID: <CAJuVHDw8tfnLuj3kpHR6UbCjZP1WsKXUXqKMm5UgJcB1i3nuJQ@mail.gmail.com>

The logo can be sent in the email like below

<img src="${url.resourcesPath}/img/sci-logo.svg" >


On Mon, Sep 10, 2018 at 12:46 PM Stian Thorgersen <sthorger at redhat.com>
wrote:

> For emails shouldn't images actually be encoded into the email itself
> rather than linked to?
>
> On Sun, 9 Sep 2018 at 19:18, vidhyadharan D <it.vidhyadharan at gmail.com>
> wrote:
>
>> Hi Experts,
>>
>> I have been working on keycloak html email. I am in need to embed  logo
>> to
>> the emails i.e. from the *themes/email/resources/img/logo.png *
>>
>>  In the login module there is a way to locate image / favicon via
>> ${url.resourcesPath}
>>
>> However in email module we dont have access to locate the resources..
>>
>> I have achieved by adding custom email template provider. If possible
>> please add these into email module because it is useful for all .
>>
>> or let me know i can provide PR.
>>
>> https://issues.jboss.org/browse/KEYCLOAK-8162
>>
>>
>> Thanks,
>> vidhya
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>

From it.vidhyadharan at gmail.com  Tue Sep 11 04:08:36 2018
From: it.vidhyadharan at gmail.com (vidhyadharan D)
Date: Tue, 11 Sep 2018 13:38:36 +0530
Subject: [keycloak-dev] Regarding
	https://issues.jboss.org/browse/KEYCLOAK-8162
In-Reply-To: <CAJuVHDw8tfnLuj3kpHR6UbCjZP1WsKXUXqKMm5UgJcB1i3nuJQ@mail.gmail.com>
References: <CAJuVHDytM+FbAxJr88BwWN1Q7rzOcppyv+mtBNs8-77vX5emuQ@mail.gmail.com>
	<CAJgngAe=O9c5SCRdQB5pxs0UhDvq5ih+PJC-1CXkx1jzpfi3Yg@mail.gmail.com>
	<CAJuVHDw8tfnLuj3kpHR6UbCjZP1WsKXUXqKMm5UgJcB1i3nuJQ@mail.gmail.com>
Message-ID: <CAJuVHDyduHe3SzDOrYD3+sUdE1CWjdhroE9_UxOu3JE0kvb2Ww@mail.gmail.com>

<img src="${hostname}/${url.resourcesPath}/img/sci-logo.svg" >


On Tue, Sep 11, 2018 at 1:37 PM vidhyadharan D <it.vidhyadharan at gmail.com>
wrote:

> The logo can be sent in the email like below
>
> <img src="${url.resourcesPath}/img/sci-logo.svg" >
>
>
> On Mon, Sep 10, 2018 at 12:46 PM Stian Thorgersen <sthorger at redhat.com>
> wrote:
>
>> For emails shouldn't images actually be encoded into the email itself
>> rather than linked to?
>>
>> On Sun, 9 Sep 2018 at 19:18, vidhyadharan D <it.vidhyadharan at gmail.com>
>> wrote:
>>
>>> Hi Experts,
>>>
>>> I have been working on keycloak html email. I am in need to embed  logo
>>> to
>>> the emails i.e. from the *themes/email/resources/img/logo.png *
>>>
>>>  In the login module there is a way to locate image / favicon via
>>> ${url.resourcesPath}
>>>
>>> However in email module we dont have access to locate the resources..
>>>
>>> I have achieved by adding custom email template provider. If possible
>>> please add these into email module because it is useful for all .
>>>
>>> or let me know i can provide PR.
>>>
>>> https://issues.jboss.org/browse/KEYCLOAK-8162
>>>
>>>
>>> Thanks,
>>> vidhya
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>

From sthorger at redhat.com  Tue Sep 11 09:08:40 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Tue, 11 Sep 2018 15:08:40 +0200
Subject: [keycloak-dev] Regarding
	https://issues.jboss.org/browse/KEYCLOAK-8162
In-Reply-To: <CAJuVHDyduHe3SzDOrYD3+sUdE1CWjdhroE9_UxOu3JE0kvb2Ww@mail.gmail.com>
References: <CAJuVHDytM+FbAxJr88BwWN1Q7rzOcppyv+mtBNs8-77vX5emuQ@mail.gmail.com>
	<CAJgngAe=O9c5SCRdQB5pxs0UhDvq5ih+PJC-1CXkx1jzpfi3Yg@mail.gmail.com>
	<CAJuVHDw8tfnLuj3kpHR6UbCjZP1WsKXUXqKMm5UgJcB1i3nuJQ@mail.gmail.com>
	<CAJuVHDyduHe3SzDOrYD3+sUdE1CWjdhroE9_UxOu3JE0kvb2Ww@mail.gmail.com>
Message-ID: <CAJgngAfazPAYV238nMnWHkcFZMB=hdJjYBQp8BCP7b-nKtMvnw@mail.gmail.com>

I was thinking inline base64 images. I don't have an issue with a PR to
make the resourceUrl available to emails, do you need it in the template or
the properties? Having it in the properties is a lot messier.

On Tue, 11 Sep 2018 at 10:08, vidhyadharan D <it.vidhyadharan at gmail.com>
wrote:

> <img src="${hostname}/${url.resourcesPath}/img/sci-logo.svg" >
>
>
> On Tue, Sep 11, 2018 at 1:37 PM vidhyadharan D <it.vidhyadharan at gmail.com>
> wrote:
>
>> The logo can be sent in the email like below
>>
>> <img src="${url.resourcesPath}/img/sci-logo.svg" >
>>
>>
>> On Mon, Sep 10, 2018 at 12:46 PM Stian Thorgersen <sthorger at redhat.com>
>> wrote:
>>
>>> For emails shouldn't images actually be encoded into the email itself
>>> rather than linked to?
>>>
>>> On Sun, 9 Sep 2018 at 19:18, vidhyadharan D <it.vidhyadharan at gmail.com>
>>> wrote:
>>>
>>>> Hi Experts,
>>>>
>>>> I have been working on keycloak html email. I am in need to embed
>>>> logo  to
>>>> the emails i.e. from the *themes/email/resources/img/logo.png *
>>>>
>>>>  In the login module there is a way to locate image / favicon via
>>>> ${url.resourcesPath}
>>>>
>>>> However in email module we dont have access to locate the resources..
>>>>
>>>> I have achieved by adding custom email template provider. If possible
>>>> please add these into email module because it is useful for all .
>>>>
>>>> or let me know i can provide PR.
>>>>
>>>> https://issues.jboss.org/browse/KEYCLOAK-8162
>>>>
>>>>
>>>> Thanks,
>>>> vidhya
>>>> _______________________________________________
>>>> keycloak-dev mailing list
>>>> keycloak-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>
>>>

From ruria at zzircon.com  Tue Sep 11 10:43:59 2018
From: ruria at zzircon.com (=?UTF-8?B?UmHDumwgVXLDrWEgRWxpY2Vz?=)
Date: Tue, 11 Sep 2018 16:43:59 +0200
Subject: [keycloak-dev] Proxy injects Role headers
Message-ID: <fad242ff-dd14-a391-94d6-4c70d5053450@zzircon.com>

Hi all,

We would like proxy to inject Role information as a header for
authorization purpouses. I think this can be easily achive by adding a
new key for "header-names" in config.json and putting the information
that already exists on the back request ...

We have a working branch for this, maybe we can pull request if is
interesting for others.

bye.
-- 

?

?

*Ra?l Ur?a El?ces*

Director

?

e-mail: ruria at zzircon.com<mailto:ruria at zzircon.com>

web:???? www.zzircon.com<http://www.zzircon.com/>

Tlf:??????? 609 721021

?

ZZircon Technologies SL

Parque Cient?fico Tecnol?gico de Cantabria

Edificio 3000 Oficina 11

c/ Isabel Torres 11

39011 Santander

?

?


/Este mensaje de correo electr?nico y sus documentos adjuntos est?n
dirigidos EXCLUSIVAMENTE a los destinatarios especificados. La
informaci?n contenida puede ser CONFIDENCIAL y / o estar LEGALMENTE
PROTEGIDA y no necesariamente refleja la opini?n de este grupo de
correo. Si usted recibe este mensaje por ERROR, por favor comun?queselo
inmediatamente al remitente y elim?nelo ya que usted NO ESTA AUTORIZADO
al uso, revelaci?n, distribuci?n, impresi?n o copia de toda o alguna
parte de la informaci?n contenida. Gracias./

? ZZircon Technologies SL- TFNO 609 721021 ???
info at zzircon.com<mailto:info at zzircon.com>

?

?

?

?

?

-------------- next part --------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: jgghhgnpkcpoojee.jpg
Type: image/jpeg
Size: 4300 bytes
Desc: not available
Url : http://lists.jboss.org/pipermail/keycloak-dev/attachments/20180911/8d25719b/attachment.jpg 

From slaskawi at redhat.com  Wed Sep 12 04:00:05 2018
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Wed, 12 Sep 2018 10:00:05 +0200
Subject: [keycloak-dev] Clustering configuration
Message-ID: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>

Hey guys,

During our weekly sync meeting, Stian asked me to look into different
options for clustering in Keycloak server. This topic has quite hot with
the context of our Docker image (see the proposed community contributions
[1][2][3]). Since we are based on WF 13, which uses JGroups 4.0.11 and has
KUBE_PING in its modules, we have a couple of options how to do it.

Before discussing different implementations, let me quickly go through the
requirements:
- We need a configuration stack that works for on-prem and cloud
deployments with OpenShift as our primary target.
- The configuration should be automatic (if it's possible). E.g. if we
discover that Keycloak is running in the container, we should use proper
discovery protocol.
- There needs to be a way to override the discovery protocol manually.

With those requirements in mind, we have a couple of implementation options
on the table:
1. Add more stacks to the configuration, e.g. openshift, azure or gcp. Then
we use the standard `-Djboss.default.jgroups.stack=<stack>` configuration
switch.
2. Provide more standalone-*.xml configuration files, e.g.
standalone-ha.xml (for on-prem) or standalone-cloud.xml.
3. Add protocols dynamically using CLI. A similar approach to what we did
for the Data Grid Cache Service [4].
4. Use MULTI_PING protocols [5][6], with multiple discovery protocols on
the same stack. This will include MPING (for multicasting), KUBE_PING (if
we can access Kubernetes API), DNS_PING (if Pods are governed by a Service).

Option #1 and #2 is somewhat similar to what we did for Infinispan [7]. It
works quite well but the configuration grows quite quickly and most of the
protocols (apart from discovery) are duplicated. On the other hand, having
separate configuration pieces for each use case is very flexible. Having in
mind that AWS cuts TCP connections, using FD_SOCK might lead to false
suspicions but on GCP for the instance, FD_SOCK works quite nicely. The CLI
option (#3), is also very flexible and probably should be implemented only
in our Docker image. This somehow follows the convention we already started
with different CLI files for different DBs [8]. Option #4 is brand new
(implemented in JGroups 4.0.8; we have 4.0.11 as you probably recall). It
has been specifically designed for this kind of use cases where we want to
gather discovery data from multiple places. Using this way, we should end
up with two stacks in standalone-ha.xml file - UDP and TCP.

I honestly need to say, that my heart goes for options #4. However, as far
as I know it hasn't been battle tested and we might get some surprises. All
other options are not as elegant as option #4 but they are used somewhere
in other projects. They are much safer options but they will add some
maintenance burden on our shoulders.

What would you suggest guys? What do you think about all this? @Rado,
@Paul, @Tristan - Do you have any plans regarding this piece in Wildfly or
Infinispan?

Thanks,
Sebastian

[1] https://github.com/jboss-dockerfiles/keycloak/pull/96
[2] https://github.com/jboss-dockerfiles/keycloak/pull/100
[3] https://github.com/jboss-dockerfiles/keycloak/pull/116
[4]
https://github.com/jboss-container-images/datagrid-7-image/blob/datagrid-services-dev/modules/os-datagrid-online-services-configuration/src/main/bash/profiles/caching-service.cli#L37
[5] http://www.jgroups.org/manual4/index.html#_multi_ping
[6] https://issues.jboss.org/browse/JGRP-2224
[7]
https://github.com/infinispan/infinispan/tree/master/server/integration/jgroups/src/main/resources/subsystem-templates
[8]
https://github.com/jboss-dockerfiles/keycloak/tree/master/server/tools/cli/databases

From it.vidhyadharan at gmail.com  Wed Sep 12 04:30:37 2018
From: it.vidhyadharan at gmail.com (vidhyadharan D)
Date: Wed, 12 Sep 2018 14:00:37 +0530
Subject: [keycloak-dev] Regarding
	https://issues.jboss.org/browse/KEYCLOAK-8162
In-Reply-To: <CAJgngAfazPAYV238nMnWHkcFZMB=hdJjYBQp8BCP7b-nKtMvnw@mail.gmail.com>
References: <CAJuVHDytM+FbAxJr88BwWN1Q7rzOcppyv+mtBNs8-77vX5emuQ@mail.gmail.com>
	<CAJgngAe=O9c5SCRdQB5pxs0UhDvq5ih+PJC-1CXkx1jzpfi3Yg@mail.gmail.com>
	<CAJuVHDw8tfnLuj3kpHR6UbCjZP1WsKXUXqKMm5UgJcB1i3nuJQ@mail.gmail.com>
	<CAJuVHDyduHe3SzDOrYD3+sUdE1CWjdhroE9_UxOu3JE0kvb2Ww@mail.gmail.com>
	<CAJgngAfazPAYV238nMnWHkcFZMB=hdJjYBQp8BCP7b-nKtMvnw@mail.gmail.com>
Message-ID: <CAJuVHDymb2h-1kOvFT=T2O2kx09D6tKG3DXWnidjJ+u7HyTo6Q@mail.gmail.com>

suppose if i move the themes from development  to production, in that case
i need to update the properties each time . where as in template variable
it is automatically calculated.


inline base64  images will suit for web emails, but for outlook it is not
supported , however logo svg will apt for all email clients.

by implementing the *${hostname}/${url.**resourcesPath}* *Custom fonts*
also served from *themes location*.

<style>
.
.
.
@font-face {
font-family: 'Roboto';
src: url('${hostname}${url.resourcesPath}/fonts/Roboto/Roboto-Light.eot');
src: local('Roboto Light'),
local('Roboto-Light'),
url('${hostname}${url.resourcesPath}/fonts/Roboto/Roboto-Light.eot?#iefix')
format('embedded-opentype'),
url('${hostname}${url.resourcesPath}/fonts/Roboto/Roboto-Light.woff2')
format('woff2'),
url('${hostname}${url.resourcesPath}/fonts/Roboto/Roboto-Light.woff') format
('woff'),
url('${hostname}${url.resourcesPath}/fonts/Roboto/Roboto-Light.ttf') format(
'truetype'),
url('${hostname}${url.resourcesPath}/fonts/Roboto/Roboto-Light.svg#Roboto')
format('svg');
font-weight: 300;
font-style: normal;
}
.
.
.

Thanks,
vidhya

On Tue, Sep 11, 2018 at 6:38 PM Stian Thorgersen <sthorger at redhat.com>
wrote:

> I was thinking inline base64 images. I don't have an issue with a PR to
> make the resourceUrl available to emails, do you need it in the template or
> the properties? Having it in the properties is a lot messier.
>
> On Tue, 11 Sep 2018 at 10:08, vidhyadharan D <it.vidhyadharan at gmail.com>
> wrote:
>
>> <img src="${hostname}/${url.resourcesPath}/img/sci-logo.svg" >
>>
>>
>> On Tue, Sep 11, 2018 at 1:37 PM vidhyadharan D <it.vidhyadharan at gmail.com>
>> wrote:
>>
>>> The logo can be sent in the email like below
>>>
>>> <img src="${url.resourcesPath}/img/sci-logo.svg" >
>>>
>>>
>>> On Mon, Sep 10, 2018 at 12:46 PM Stian Thorgersen <sthorger at redhat.com>
>>> wrote:
>>>
>>>> For emails shouldn't images actually be encoded into the email itself
>>>> rather than linked to?
>>>>
>>>> On Sun, 9 Sep 2018 at 19:18, vidhyadharan D <it.vidhyadharan at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Experts,
>>>>>
>>>>> I have been working on keycloak html email. I am in need to embed
>>>>> logo  to
>>>>> the emails i.e. from the *themes/email/resources/img/logo.png *
>>>>>
>>>>>  In the login module there is a way to locate image / favicon via
>>>>> ${url.resourcesPath}
>>>>>
>>>>> However in email module we dont have access to locate the resources..
>>>>>
>>>>> I have achieved by adding custom email template provider. If possible
>>>>> please add these into email module because it is useful for all .
>>>>>
>>>>> or let me know i can provide PR.
>>>>>
>>>>> https://issues.jboss.org/browse/KEYCLOAK-8162
>>>>>
>>>>>
>>>>> Thanks,
>>>>> vidhya
>>>>> _______________________________________________
>>>>> keycloak-dev mailing list
>>>>> keycloak-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>
>>>>

From Sebastian.Schuster at bosch-si.com  Wed Sep 12 05:17:22 2018
From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST-CSS/BSV-OS))
Date: Wed, 12 Sep 2018 09:17:22 +0000
Subject: [keycloak-dev] Clustering configuration
In-Reply-To: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>
References: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>
Message-ID: <ead49152b2af42d48126abdf2ff1f90c@bosch-si.com>

Hi Sebastian! :)

What about just using JDBC_PING as a default? It works in any environment, it does not add an additional dependency as Keycloak does not do much without DB anyways.
The only problem we are currently having is that the graceful shutdown does not work since the dependency of the Jgroups subsystem to the DB is not identified correctly.

Best regards,
Sebastian 

Mit freundlichen Gr??en / Best regards

Dr.-Ing.  Sebastian Schuster

Engineering and Support (INST/ESY1) 
Bosch?Software Innovations?GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B 
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn 




-----Original Message-----
From: keycloak-dev-bounces at lists.jboss.org <keycloak-dev-bounces at lists.jboss.org> On Behalf Of Sebastian Laskawiec
Sent: Mittwoch, 12. September 2018 10:00
To: keycloak-dev <keycloak-dev at lists.jboss.org>
Cc: Bela Ban <bban at redhat.com>; Radoslav Husar <rhusar at redhat.com>; Tarrant, Tristan <ttarrant at redhat.com>; Paul Ferraro <paul.ferraro at redhat.com>
Subject: [keycloak-dev] Clustering configuration

Hey guys,

During our weekly sync meeting, Stian asked me to look into different options for clustering in Keycloak server. This topic has quite hot with the context of our Docker image (see the proposed community contributions [1][2][3]). Since we are based on WF 13, which uses JGroups 4.0.11 and has KUBE_PING in its modules, we have a couple of options how to do it.

Before discussing different implementations, let me quickly go through the
requirements:
- We need a configuration stack that works for on-prem and cloud deployments with OpenShift as our primary target.
- The configuration should be automatic (if it's possible). E.g. if we discover that Keycloak is running in the container, we should use proper discovery protocol.
- There needs to be a way to override the discovery protocol manually.

With those requirements in mind, we have a couple of implementation options on the table:
1. Add more stacks to the configuration, e.g. openshift, azure or gcp. Then we use the standard `-Djboss.default.jgroups.stack=<stack>` configuration switch.
2. Provide more standalone-*.xml configuration files, e.g.
standalone-ha.xml (for on-prem) or standalone-cloud.xml.
3. Add protocols dynamically using CLI. A similar approach to what we did for the Data Grid Cache Service [4].
4. Use MULTI_PING protocols [5][6], with multiple discovery protocols on the same stack. This will include MPING (for multicasting), KUBE_PING (if we can access Kubernetes API), DNS_PING (if Pods are governed by a Service).

Option #1 and #2 is somewhat similar to what we did for Infinispan [7]. It works quite well but the configuration grows quite quickly and most of the protocols (apart from discovery) are duplicated. On the other hand, having separate configuration pieces for each use case is very flexible. Having in mind that AWS cuts TCP connections, using FD_SOCK might lead to false suspicions but on GCP for the instance, FD_SOCK works quite nicely. The CLI option (#3), is also very flexible and probably should be implemented only in our Docker image. This somehow follows the convention we already started with different CLI files for different DBs [8]. Option #4 is brand new (implemented in JGroups 4.0.8; we have 4.0.11 as you probably recall). It has been specifically designed for this kind of use cases where we want to gather discovery data from multiple places. Using this way, we should end up with two stacks in standalone-ha.xml file - UDP and TCP.

I honestly need to say, that my heart goes for options #4. However, as far as I know it hasn't been battle tested and we might get some surprises. All other options are not as elegant as option #4 but they are used somewhere in other projects. They are much safer options but they will add some maintenance burden on our shoulders.

What would you suggest guys? What do you think about all this? @Rado, @Paul, @Tristan - Do you have any plans regarding this piece in Wildfly or Infinispan?

Thanks,
Sebastian

[1] https://github.com/jboss-dockerfiles/keycloak/pull/96
[2] https://github.com/jboss-dockerfiles/keycloak/pull/100
[3] https://github.com/jboss-dockerfiles/keycloak/pull/116
[4]
https://github.com/jboss-container-images/datagrid-7-image/blob/datagrid-services-dev/modules/os-datagrid-online-services-configuration/src/main/bash/profiles/caching-service.cli#L37
[5] http://www.jgroups.org/manual4/index.html#_multi_ping
[6] https://issues.jboss.org/browse/JGRP-2224
[7]
https://github.com/infinispan/infinispan/tree/master/server/integration/jgroups/src/main/resources/subsystem-templates
[8]
https://github.com/jboss-dockerfiles/keycloak/tree/master/server/tools/cli/databases
_______________________________________________
keycloak-dev mailing list
keycloak-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev


From thomas.darimont at googlemail.com  Wed Sep 12 05:41:02 2018
From: thomas.darimont at googlemail.com (Thomas Darimont)
Date: Wed, 12 Sep 2018 11:41:02 +0200
Subject: [keycloak-dev] Clustering configuration
In-Reply-To: <ead49152b2af42d48126abdf2ff1f90c@bosch-si.com>
References: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>
	<ead49152b2af42d48126abdf2ff1f90c@bosch-si.com>
Message-ID: <CAK-7U1gftKqXzh1R3fm7k2qU71eDwJDP1BOjt+uHz5feuZ-b8w@mail.gmail.com>

Hi all,

while you are mentioning JDBC_PING it seems that the configuration somewhat
changed between 4.2.1 and 4.4.0 probably due to the wildfly /infinispan
upgrade. The keycloak Helm chart for Kubernetes just stumbled upon this
while trying to upgrade to 4.4.0.Final.
See:
https://github.com/helm/charts/pull/7650#issuecomment-420174274

BTW. another configuration option would be to use TCP unicast with a set of
initial hosts for discovery. This is useful in scenarios where multiple
network-segments are involved that don't support UDP multicast and
jdbc_ping cannot be used (... for what ever reason).

Cheers,
Thomas

Schuster Sebastian (INST-CSS/BSV-OS) <Sebastian.Schuster at bosch-si.com>
schrieb am Mi., 12. Sep. 2018, 11:25:

> Hi Sebastian! :)
>
> What about just using JDBC_PING as a default? It works in any environment,
> it does not add an additional dependency as Keycloak does not do much
> without DB anyways.
> The only problem we are currently having is that the graceful shutdown
> does not work since the dependency of the Jgroups subsystem to the DB is
> not identified correctly.
>
> Best regards,
> Sebastian
>
> Mit freundlichen Gr??en / Best regards
>
> Dr.-Ing.  Sebastian Schuster
>
> Engineering and Support (INST/ESY1)
> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
> GERMANY | www.bosch-si.com
> Tel. +49 30 726112-485 | Fax +49 30 726112-100 |
> Sebastian.Schuster at bosch-si.com
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr.
> Stefan Ferber, Michael Hahn
>
>
>
>
> -----Original Message-----
> From: keycloak-dev-bounces at lists.jboss.org <
> keycloak-dev-bounces at lists.jboss.org> On Behalf Of Sebastian Laskawiec
> Sent: Mittwoch, 12. September 2018 10:00
> To: keycloak-dev <keycloak-dev at lists.jboss.org>
> Cc: Bela Ban <bban at redhat.com>; Radoslav Husar <rhusar at redhat.com>;
> Tarrant, Tristan <ttarrant at redhat.com>; Paul Ferraro <
> paul.ferraro at redhat.com>
> Subject: [keycloak-dev] Clustering configuration
>
> Hey guys,
>
> During our weekly sync meeting, Stian asked me to look into different
> options for clustering in Keycloak server. This topic has quite hot with
> the context of our Docker image (see the proposed community contributions
> [1][2][3]). Since we are based on WF 13, which uses JGroups 4.0.11 and has
> KUBE_PING in its modules, we have a couple of options how to do it.
>
> Before discussing different implementations, let me quickly go through the
> requirements:
> - We need a configuration stack that works for on-prem and cloud
> deployments with OpenShift as our primary target.
> - The configuration should be automatic (if it's possible). E.g. if we
> discover that Keycloak is running in the container, we should use proper
> discovery protocol.
> - There needs to be a way to override the discovery protocol manually.
>
> With those requirements in mind, we have a couple of implementation
> options on the table:
> 1. Add more stacks to the configuration, e.g. openshift, azure or gcp.
> Then we use the standard `-Djboss.default.jgroups.stack=<stack>`
> configuration switch.
> 2. Provide more standalone-*.xml configuration files, e.g.
> standalone-ha.xml (for on-prem) or standalone-cloud.xml.
> 3. Add protocols dynamically using CLI. A similar approach to what we did
> for the Data Grid Cache Service [4].
> 4. Use MULTI_PING protocols [5][6], with multiple discovery protocols on
> the same stack. This will include MPING (for multicasting), KUBE_PING (if
> we can access Kubernetes API), DNS_PING (if Pods are governed by a Service).
>
> Option #1 and #2 is somewhat similar to what we did for Infinispan [7]. It
> works quite well but the configuration grows quite quickly and most of the
> protocols (apart from discovery) are duplicated. On the other hand, having
> separate configuration pieces for each use case is very flexible. Having in
> mind that AWS cuts TCP connections, using FD_SOCK might lead to false
> suspicions but on GCP for the instance, FD_SOCK works quite nicely. The CLI
> option (#3), is also very flexible and probably should be implemented only
> in our Docker image. This somehow follows the convention we already started
> with different CLI files for different DBs [8]. Option #4 is brand new
> (implemented in JGroups 4.0.8; we have 4.0.11 as you probably recall). It
> has been specifically designed for this kind of use cases where we want to
> gather discovery data from multiple places. Using this way, we should end
> up with two stacks in standalone-ha.xml file - UDP and TCP.
>
> I honestly need to say, that my heart goes for options #4. However, as far
> as I know it hasn't been battle tested and we might get some surprises. All
> other options are not as elegant as option #4 but they are used somewhere
> in other projects. They are much safer options but they will add some
> maintenance burden on our shoulders.
>
> What would you suggest guys? What do you think about all this? @Rado,
> @Paul, @Tristan - Do you have any plans regarding this piece in Wildfly or
> Infinispan?
>
> Thanks,
> Sebastian
>
> [1] https://github.com/jboss-dockerfiles/keycloak/pull/96
> [2] https://github.com/jboss-dockerfiles/keycloak/pull/100
> [3] https://github.com/jboss-dockerfiles/keycloak/pull/116
> [4]
>
> https://github.com/jboss-container-images/datagrid-7-image/blob/datagrid-services-dev/modules/os-datagrid-online-services-configuration/src/main/bash/profiles/caching-service.cli#L37
> [5] http://www.jgroups.org/manual4/index.html#_multi_ping
> [6] https://issues.jboss.org/browse/JGRP-2224
> [7]
>
> https://github.com/infinispan/infinispan/tree/master/server/integration/jgroups/src/main/resources/subsystem-templates
> [8]
>
> https://github.com/jboss-dockerfiles/keycloak/tree/master/server/tools/cli/databases
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

From Sebastian.Schuster at bosch-si.com  Wed Sep 12 05:56:48 2018
From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST-CSS/BSV-OS))
Date: Wed, 12 Sep 2018 09:56:48 +0000
Subject: [keycloak-dev] Clustering configuration
In-Reply-To: <CAK-7U1gftKqXzh1R3fm7k2qU71eDwJDP1BOjt+uHz5feuZ-b8w@mail.gmail.com>
References: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>
	<ead49152b2af42d48126abdf2ff1f90c@bosch-si.com>
	<CAK-7U1gftKqXzh1R3fm7k2qU71eDwJDP1BOjt+uHz5feuZ-b8w@mail.gmail.com>
Message-ID: <a8a8ada680c24b6e910d2b8bf8c3b07a@bosch-si.com>

Guess what, our JDBC_PING configuration not working with 4.4.0.Final is what I am currently working on.
Thanks Thomas!

Best regards,
Sebastian

Mit freundlichen Gr??en / Best regards

Dr.-Ing. Sebastian Schuster

Engineering and Support (INST/ESY1)
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn



From: Thomas Darimont <thomas.darimont at googlemail.com>
Sent: Mittwoch, 12. September 2018 11:41
To: Schuster Sebastian (INST-CSS/BSV-OS) <Sebastian.Schuster at bosch-si.com>
Cc: Sebastian Laskawiec <slaskawi at redhat.com>; keycloak-dev <keycloak-dev at lists.jboss.org>; Bela Ban <bban at redhat.com>; Radoslav Husar <rhusar at redhat.com>; Tarrant, Tristan <ttarrant at redhat.com>; Paul Ferraro <paul.ferraro at redhat.com>
Subject: Re: [keycloak-dev] Clustering configuration

Hi all,

while you are mentioning JDBC_PING it seems that the configuration somewhat changed between 4.2.1 and 4.4.0 probably due to the wildfly /infinispan upgrade. The keycloak Helm chart for Kubernetes just stumbled upon this while trying to upgrade to 4.4.0.Final.
See:
https://github.com/helm/charts/pull/7650#issuecomment-420174274

BTW. another configuration option would be to use TCP unicast with a set of initial hosts for discovery. This is useful in scenarios where multiple network-segments are involved that don't support UDP multicast and jdbc_ping cannot be used (... for what ever reason).

Cheers,
Thomas

Schuster Sebastian (INST-CSS/BSV-OS) <Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>> schrieb am Mi., 12. Sep. 2018, 11:25:
Hi Sebastian! :)

What about just using JDBC_PING as a default? It works in any environment, it does not add an additional dependency as Keycloak does not do much without DB anyways.
The only problem we are currently having is that the graceful shutdown does not work since the dependency of the Jgroups subsystem to the DB is not identified correctly.

Best regards,
Sebastian

Mit freundlichen Gr??en / Best regards

Dr.-Ing.  Sebastian Schuster

Engineering and Support (INST/ESY1)
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn




-----Original Message-----
From: keycloak-dev-bounces at lists.jboss.org<mailto:keycloak-dev-bounces at lists.jboss.org> <keycloak-dev-bounces at lists.jboss.org<mailto:keycloak-dev-bounces at lists.jboss.org>> On Behalf Of Sebastian Laskawiec
Sent: Mittwoch, 12. September 2018 10:00
To: keycloak-dev <keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>>
Cc: Bela Ban <bban at redhat.com<mailto:bban at redhat.com>>; Radoslav Husar <rhusar at redhat.com<mailto:rhusar at redhat.com>>; Tarrant, Tristan <ttarrant at redhat.com<mailto:ttarrant at redhat.com>>; Paul Ferraro <paul.ferraro at redhat.com<mailto:paul.ferraro at redhat.com>>
Subject: [keycloak-dev] Clustering configuration

Hey guys,

During our weekly sync meeting, Stian asked me to look into different options for clustering in Keycloak server. This topic has quite hot with the context of our Docker image (see the proposed community contributions [1][2][3]). Since we are based on WF 13, which uses JGroups 4.0.11 and has KUBE_PING in its modules, we have a couple of options how to do it.

Before discussing different implementations, let me quickly go through the
requirements:
- We need a configuration stack that works for on-prem and cloud deployments with OpenShift as our primary target.
- The configuration should be automatic (if it's possible). E.g. if we discover that Keycloak is running in the container, we should use proper discovery protocol.
- There needs to be a way to override the discovery protocol manually.

With those requirements in mind, we have a couple of implementation options on the table:
1. Add more stacks to the configuration, e.g. openshift, azure or gcp. Then we use the standard `-Djboss.default.jgroups.stack=<stack>` configuration switch.
2. Provide more standalone-*.xml configuration files, e.g.
standalone-ha.xml (for on-prem) or standalone-cloud.xml.
3. Add protocols dynamically using CLI. A similar approach to what we did for the Data Grid Cache Service [4].
4. Use MULTI_PING protocols [5][6], with multiple discovery protocols on the same stack. This will include MPING (for multicasting), KUBE_PING (if we can access Kubernetes API), DNS_PING (if Pods are governed by a Service).

Option #1 and #2 is somewhat similar to what we did for Infinispan [7]. It works quite well but the configuration grows quite quickly and most of the protocols (apart from discovery) are duplicated. On the other hand, having separate configuration pieces for each use case is very flexible. Having in mind that AWS cuts TCP connections, using FD_SOCK might lead to false suspicions but on GCP for the instance, FD_SOCK works quite nicely. The CLI option (#3), is also very flexible and probably should be implemented only in our Docker image. This somehow follows the convention we already started with different CLI files for different DBs [8]. Option #4 is brand new (implemented in JGroups 4.0.8; we have 4.0.11 as you probably recall). It has been specifically designed for this kind of use cases where we want to gather discovery data from multiple places. Using this way, we should end up with two stacks in standalone-ha.xml file - UDP and TCP.

I honestly need to say, that my heart goes for options #4. However, as far as I know it hasn't been battle tested and we might get some surprises. All other options are not as elegant as option #4 but they are used somewhere in other projects. They are much safer options but they will add some maintenance burden on our shoulders.

What would you suggest guys? What do you think about all this? @Rado, @Paul, @Tristan - Do you have any plans regarding this piece in Wildfly or Infinispan?

Thanks,
Sebastian

[1] https://github.com/jboss-dockerfiles/keycloak/pull/96
[2] https://github.com/jboss-dockerfiles/keycloak/pull/100
[3] https://github.com/jboss-dockerfiles/keycloak/pull/116
[4]
https://github.com/jboss-container-images/datagrid-7-image/blob/datagrid-services-dev/modules/os-datagrid-online-services-configuration/src/main/bash/profiles/caching-service.cli#L37
[5] http://www.jgroups.org/manual4/index.html#_multi_ping
[6] https://issues.jboss.org/browse/JGRP-2224
[7]
https://github.com/infinispan/infinispan/tree/master/server/integration/jgroups/src/main/resources/subsystem-templates
[8]
https://github.com/jboss-dockerfiles/keycloak/tree/master/server/tools/cli/databases
_______________________________________________
keycloak-dev mailing list
keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-dev

_______________________________________________
keycloak-dev mailing list
keycloak-dev at lists.jboss.org<mailto:keycloak-dev at lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-dev

From cedric at couralet.eu  Wed Sep 12 07:17:55 2018
From: cedric at couralet.eu (=?utf-8?q?cedric=40couralet=2Eeu?=)
Date: Wed, 12 Sep 2018 13:17:55 +0200
Subject: [keycloak-dev] =?utf-8?q?Clustering_configuration?=
In-Reply-To: <ead49152b2af42d48126abdf2ff1f90c@bosch-si.com>
Message-ID: <608c-5b98f600-7-769a4980@46765112>

Hi all, 

Le Mercredi, Septembre 12, 2018 11:17 CEST, "Schuster Sebastian (INST-CSS/BSV-OS)" <Sebastian.Schuster at bosch-si.com> a ?crit: 
 
> Hi Sebastian! :)
> 
> What about just using JDBC_PING as a default? It works in any environment, it does not add an additional dependency as Keycloak does not do much without DB anyways.

+1 for JDBC_PING as an option. I am using Mesos at the moment, and JDBC_PING is the only option for me. At minimal, documenting how to add the configuration in a custom keycloak docker image would be great.

> The only problem we are currently having is that the graceful shutdown does not work since the dependency of the Jgroups subsystem to the DB is not identified correctly.
> 
> Best regards,
> Sebastian 
>



From cedric at couralet.eu  Wed Sep 12 07:33:16 2018
From: cedric at couralet.eu (=?utf-8?q?cedric=40couralet=2Eeu?=)
Date: Wed, 12 Sep 2018 13:33:16 +0200
Subject: [keycloak-dev] =?utf-8?q?Clustering_configuration?=
In-Reply-To: <a8a8ada680c24b6e910d2b8bf8c3b07a@bosch-si.com>
Message-ID: <6ffb-5b98f980-7-33393680@14777610>

Hi Sebastian,
 
Le Mercredi, Septembre 12, 2018 11:56 CEST, "Schuster Sebastian (INST-CSS/BSV-OS)" <Sebastian.Schuster at bosch-si.com> a ?crit: 
 
> Guess what, our JDBC_PING configuration not working with 4.4.0.Final is what I am currently working on.

If it helps, I had some difficulties configuring JDBC_PING on keycloak 4.4.0.Final. 

My final working configuration is :
       <subsystem xmlns="urn:jboss:domain:jgroups:6.0">
            <channels default="ee">
                <channel name="ee" stack="tcpping"/>
            </channels>
            <stacks>
                <stack name="tcpping">
                    <transport type="TCP" socket-binding="jgroups-tcp">
                        <property name="external_addr">
                            ${jgroups.bind.address:127.0.0.1}
                        </property>
                        <property name="bind_addr">
                            ${jgroups.bind_addr:SITE_LOCAL}
                        </property>
                    </transport>
                    <jdbc-protocol type="JDBC_PING" data-source="KeycloakDS">
                        <property name="initialize_sql">
                            CREATE TABLE IF NOT EXISTS JGROUPSPING (own_addr varchar(200) NOT NULL,bind_addr varchar(200) NOT NULL,created timestamp NOT NULL,cluster_name varchar(200) NOT NULL,ping_data BYTEA,constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name))
                        </property>
                        <property name="insert_single_sql">
                            INSERT INTO JGROUPSPING (own_addr, bind_addr, created, cluster_name, ping_data) values (?,'${jgroups.bind.address:127.0.0.1}',NOW(), ?, ?)
                        </property>
                        <property name="delete_single_sql">
                            DELETE FROM JGROUPSPING WHERE own_addr=? AND cluster_name=?
                        </property>
                        <property name="select_all_pingdata_sql">
                            SELECT ping_data FROM JGROUPSPING WHERE cluster_name=?;
                        </property>
                    </jdbc-protocol>
                    <protocol type="MERGE3"/>
                    <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd">
                        <property name="external_addr">
                            ${jgroups.bind.address:127.0.0.1}
                        </property>
                    </protocol>
                    <protocol type="FD"/>
                    <protocol type="VERIFY_SUSPECT"/>
                    <protocol type="pbcast.NAKACK2"/>
                    <protocol type="UNICAST3"/>
                    <protocol type="pbcast.STABLE"/>
                    <protocol type="pbcast.GMS"/>
                    <protocol type="MFC"/>
                    <protocol type="FRAG2"/>
                </stack>
            </stacks>
        </subsystem>


I do it in my own docker image where I change the configuration with jboss-cli (as in the officilal) with this file :
/subsystem=infinispan/cache-container=keycloak/distributed-cache=sessions:remove
/subsystem=infinispan/cache-container=keycloak/replicated-cache=sessions:add()
/subsystem=infinispan/cache-container=keycloak/replicated-cache=sessions:write-attribute(name="mode",value="SYNC")

/subsystem=infinispan/cache-container=keycloak/distributed-cache=authenticationSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=clientSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineClientSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures:remove

/subsystem=infinispan/cache-container=keycloak/distributed-cache=authenticationSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=clientSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineClientSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures:add(mode="SYNC",owners="2")

/subsystem=jgroups/stack=tcpping:add()
/subsystem=jgroups/stack=tcpping/transport=TCP:add(socket-binding=jgroups-tcp)
/subsystem=jgroups/stack=tcpping/transport=TCP/property=external_addr:add(value=${jgroups.bind.address:127.0.0.1})
/subsystem=jgroups/stack=tcpping/transport=TCP/property=bind_addr:add(value=${jgroups.bind_addr:SITE_LOCAL})


/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING:add(data-source="KeycloakDS", properties=[initialize_sql="CREATE TABLE IF NOT EXISTS JGROUPSPING (own_addr varchar(200) NOT NULL,bind_addr varchar(200) NOT NULL,created timestamp NOT NULL,cluster_name varchar(200) NOT NULL,ping_data BYTEA,constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name))",insert_single_sql="INSERT INTO JGROUPSPING (own_addr, bind_addr, created, cluster_name, ping_data) values (?,'${jgroups.bind.address:127.0.0.1}',NOW(), ?, ?)",delete_single_sql="DELETE FROM JGROUPSPING WHERE own_addr=? AND cluster_name=?",select_all_pingdata_sql="SELECT ping_data FROM JGROUPSPING WHERE cluster_name=?;"])

/subsystem=jgroups/stack=tcpping/protocol=MERGE3:add()
/subsystem=jgroups/stack=tcpping:add-protocol(type="FD_SOCK",socket-binding="jgroups-tcp-fd")
/subsystem=jgroups/stack=tcpping/protocol=FD_SOCK/property=external_addr:add(value=${jgroups.bind.address:127.0.0.1})
/subsystem=jgroups/stack=tcpping/protocol=FD:add()
/subsystem=jgroups/stack=tcpping/protocol=VERIFY_SUSPECT:add()
/subsystem=jgroups/stack=tcpping/protocol=pbcast.NAKACK2:add()
/subsystem=jgroups/stack=tcpping/protocol=UNICAST3:add()
/subsystem=jgroups/stack=tcpping/protocol=pbcast.STABLE:add()
/subsystem=jgroups/stack=tcpping/protocol=pbcast.GMS:add()
/subsystem=jgroups/stack=tcpping/protocol=MFC:add()
/subsystem=jgroups/stack=tcpping/protocol=FRAG2:add()

/subsystem=jgroups/channel=ee:remove
/subsystem=jgroups/channel=ee:add(stack=tcpping)
/subsystem=jgroups:write-attribute(name=default-channel, value=ee)

/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp:write-attribute(name="interface",value="private")
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd:add()
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd:write-attribute(name="interface",value="private")
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd:write-attribute(name="port",value="57600")


/subsystem=jgroups/stack=tcp:remove
/subsystem=jgroups/stack=udp:remove


I am not sure all is good in this but it works (in my environment :) ).

(the difficulties I had was that if the cli file is like :
...
	/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING:add(data-source="KeycloakDS")
	/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING/property=datasource_jndi_name:add(value=java:jboss/datasources/KeycloakDS)
/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING/property=otherproperty:add(value=other_value)
...

the configuration in xml is :
  <protocol type="org.apache.jgroups.JDBC_PING" >

Which doesn't work (don't know why). There is not a lot of documentation on this, so I'm listening to all suggestions.

Cheers,
C?dric Couralet



From bruno at abstractj.org  Wed Sep 12 17:42:18 2018
From: bruno at abstractj.org (Bruno Oliveira)
Date: Wed, 12 Sep 2018 18:42:18 -0300
Subject: [keycloak-dev] Removal of user-storage-jpa and user-storage-simple
Message-ID: <CAM5SUC5-utpxtUxEneQbs8Fvzpfx7ortTg317eVTm8MrdJEuLg@mail.gmail.com>

Good night,

Now that we have user-storage-jpa and user-storage-simple examples
into the Quickstarts repository
(https://github.com/keycloak/keycloak-quickstarts). I would like to
remove these projects from Keycloak repository
(https://github.com/keycloak/keycloak/tree/master/examples/providers).

Any concerns about this?

-- 
- abstractj

From slaskawi at redhat.com  Thu Sep 13 03:37:47 2018
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Thu, 13 Sep 2018 09:37:47 +0200
Subject: [keycloak-dev] Clustering configuration
In-Reply-To: <bc68249e-fb04-026c-b002-2a91f6937ad1@redhat.com>
References: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>
	<CAF5qqoMqsjGbe1ZDeU4wy=qt_kY7=wQW1X7XbtMiXE-Q2bEeKA@mail.gmail.com>
	<bc68249e-fb04-026c-b002-2a91f6937ad1@redhat.com>
Message-ID: <CAA9R9O72q6ho-pdDLx4pk1ONTY1S1oSmQAYkoYJZRRH5p=Oyag@mail.gmail.com>

Answering a several email in a row. Please forgive me for not responding
each one individually.

Keycloak is actually a web app (rather than a separate app as Infiispan,
which is more forked of Wildfly rather than layered). We already use a
database that needs to be somewhere there (either embedded H2, MySQL or
whatever). Moreover, we also define our data source - KeycloakDS. I also
expect Keycloak clusters to be rather small, just 2-3 servers for HA.

Having all this in mind, using JDBC_PING might not be a bad idea, I think.
I guess we are lucky since we can assume that we already have an SQL
database running. @Bela Ban <bban at redhat.com> I've also seen you fixed the
crashed members issue mentioned in [1] in 4.0.10. We're on 4.0.11, so we
should also be good to go.
However there are a few limitations that are worth mentioning and I would
like to request some comments from @Marek Posolda <mposolda at redhat.com>, @Stian
Thorgersen <stian at redhat.com> and @Hynek Mlnarik <hmlnarik at redhat.com>:
- This makes our dependency to the database even stronger. If we ever
decided to change the storage, this issue will get back to us like a
boomerang.
- When considering cross site replication (with database replication turned
on), I guess we will need a separate, not-replicated table for discovery
per each site. Otherwise, sites might override each other by writing into
the same row in the same table.
- We have a few data base implementation that we can work with (H2, MySQL,
PostgreSQL and others but that requires additional drivers). The JDBC_PING
uses SQL queries so we would need to write them in such a way, that they
always work regardless to the DB implementation. Since this is at the
configuration level, we can not use JPA. Those will need to be hand-crafted
queries. But I guess it shouldn't be a big deal.

The MULTI_PING also seems very interesting. Thanks a lot for all the useful
info! @Paul Ferraro <paul.ferraro at redhat.com>, could you please tell me
when do you plan to implement the multi discovery protocol behavior you
mentioned in your email? In which Wildfly version this feature will be
available?

Thanks,
Sebastian

PS - If you guys spot any problems with JDBC_PING and current version of
Keycloak, please create a JIRA for us -
https://issues.jboss.org/browse/KEYCLOAK

[1] https://issues.jboss.org/browse/JGRP-2245

On Wed, Sep 12, 2018 at 5:10 PM Bela Ban <bban at redhat.com> wrote:

>
>
> On 12/09/18 14:46, Paul Ferraro wrote:
> > Hi Sebastian,
> >
> > We've planned to support MULTI_PING in WF by adding it to the stack
> > automatically in the event that a stack contains multiple discovery
> > protocols [1].  We want to keep this hidden from users, since JGroups
> > plans to support multiple discovery protocols transparently [2].
>
> As I said before, I don't like to add hidden stuff, or else
> configuration != runtime. However, in this specific case, I agree that
> this may be a good interim solution until we have multiple discovery
> protocols without the need for MULTI_PING.
>
> > Since we haven't had time to do this yet, we have not yet performed
> > any testing of MULTI_PING, nor do I know of any users already using
> > this.
>
> Correct. I haven't heard of any users, either.
>
> Next steps wrt discovery are me thinking about multiple discovery
> protocols without MULTI_PING and perhaps a more reactive design
> (FIND_MBRS_ASYNC) involving CompletableFutures, as suggested by Dan.
>
> > If you don't have the bandwidth to toy with MULTI_PING, I would
> > strongly encourage you to use option #3.  Configuration XML is
> > permitted to change dramatically between schema versions, and can be a
> > nightmare for XML manipulation scripts, while the EAP/WF management
> > CLI is effectively a stable API, where backwards compatibility (for a
> > specific version range of the model) is a requirement.  Rado recently
> > refactored our testsuite to uses CLI scripts for modifying
> > configuration (instead of XML manipulation) and it has considerably
> > reduced the complexity of our testsuite.  I can't imagine ever going
> > back.
> >
> > CLI scripts for WF/EAP's jgroups subsystem would look considerably
> > simpler than the scripts you posted above (for the datagrid-jgroups
> > subsystem).  Specifically:
> > 1. Rather than remove and redefine an entire protocol stack, you can
> > remove a specific protocol and insert an new one in the stack at a
> > specific index.
> > 2. pbcast.NAKACK2's use_mcast_xmit attribute is auto-disabled when the
> > transport does not support multicast, so this property never needs to
> > be set explicitly.
> > 3. All clustering operations support the
> > {allow-resource-service-restart=true} operation header to avoid a full
> > server reload by restarting only affected services.  Batching
> > (especially when containing remove/add operation pairs) is encouraged
> > to eliminate redundant service restarts.
> >
> > [1] https://issues.jboss.org/browse/WFLY-9723
> > [2] https://issues.jboss.org/browse/JGRP-2230
> > On Wed, Sep 12, 2018 at 4:00 AM Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
> >>
> >> Hey guys,
> >>
> >> During our weekly sync meeting, Stian asked me to look into different
> options for clustering in Keycloak server. This topic has quite hot with
> the context of our Docker image (see the proposed community contributions
> [1][2][3]). Since we are based on WF 13, which uses JGroups 4.0.11 and has
> KUBE_PING in its modules, we have a couple of options how to do it.
> >>
> >> Before discussing different implementations, let me quickly go through
> the requirements:
> >> - We need a configuration stack that works for on-prem and cloud
> deployments with OpenShift as our primary target.
> >> - The configuration should be automatic (if it's possible). E.g. if we
> discover that Keycloak is running in the container, we should use proper
> discovery protocol.
> >> - There needs to be a way to override the discovery protocol manually.
> >>
> >> With those requirements in mind, we have a couple of implementation
> options on the table:
> >> 1. Add more stacks to the configuration, e.g. openshift, azure or gcp.
> Then we use the standard `-Djboss.default.jgroups.stack=<stack>`
> configuration switch.
> >> 2. Provide more standalone-*.xml configuration files, e.g.
> standalone-ha.xml (for on-prem) or standalone-cloud.xml.
> >> 3. Add protocols dynamically using CLI. A similar approach to what we
> did for the Data Grid Cache Service [4].
> >> 4. Use MULTI_PING protocols [5][6], with multiple discovery protocols
> on the same stack. This will include MPING (for multicasting), KUBE_PING
> (if we can access Kubernetes API), DNS_PING (if Pods are governed by a
> Service).
> >>
> >> Option #1 and #2 is somewhat similar to what we did for Infinispan [7].
> It works quite well but the configuration grows quite quickly and most of
> the protocols (apart from discovery) are duplicated. On the other hand,
> having separate configuration pieces for each use case is very flexible.
> Having in mind that AWS cuts TCP connections, using FD_SOCK might lead to
> false suspicions but on GCP for the instance, FD_SOCK works quite nicely.
> The CLI option (#3), is also very flexible and probably should be
> implemented only in our Docker image. This somehow follows the convention
> we already started with different CLI files for different DBs [8]. Option
> #4 is brand new (implemented in JGroups 4.0.8; we have 4.0.11 as you
> probably recall). It has been specifically designed for this kind of use
> cases where we want to gather discovery data from multiple places. Using
> this way, we should end up with two stacks in standalone-ha.xml file - UDP
> and TCP.
> >>
> >> I honestly need to say, that my heart goes for options #4. However, as
> far as I know it hasn't been battle tested and we might get some surprises.
> All other options are not as elegant as option #4 but they are used
> somewhere in other projects. They are much safer options but they will add
> some maintenance burden on our shoulders.
> >>
> >> What would you suggest guys? What do you think about all this? @Rado,
> @Paul, @Tristan - Do you have any plans regarding this piece in Wildfly or
> Infinispan?
> >>
> >> Thanks,
> >> Sebastian
> >>
> >> [1] https://github.com/jboss-dockerfiles/keycloak/pull/96
> >> [2] https://github.com/jboss-dockerfiles/keycloak/pull/100
> >> [3] https://github.com/jboss-dockerfiles/keycloak/pull/116
> >> [4]
> https://github.com/jboss-container-images/datagrid-7-image/blob/datagrid-services-dev/modules/os-datagrid-online-services-configuration/src/main/bash/profiles/caching-service.cli#L37
> >> [5] http://www.jgroups.org/manual4/index.html#_multi_ping
> >> [6] https://issues.jboss.org/browse/JGRP-2224
> >> [7]
> https://github.com/infinispan/infinispan/tree/master/server/integration/jgroups/src/main/resources/subsystem-templates
> >> [8]
> https://github.com/jboss-dockerfiles/keycloak/tree/master/server/tools/cli/databases
>
> --
> Bela Ban | http://www.jgroups.org
>
>

From sthorger at redhat.com  Thu Sep 13 04:16:30 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Thu, 13 Sep 2018 10:16:30 +0200
Subject: [keycloak-dev] Clustering configuration
In-Reply-To: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>
References: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>
Message-ID: <CAJgngAeM=Tw9NokDNN2OpeL5NFo-49EzyHp26ca3ckf669k0hw@mail.gmail.com>

On Wed, 12 Sep 2018 at 10:09, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

> Hey guys,
>
> During our weekly sync meeting, Stian asked me to look into different
> options for clustering in Keycloak server. This topic has quite hot with
> the context of our Docker image (see the proposed community contributions
> [1][2][3]). Since we are based on WF 13, which uses JGroups 4.0.11 and has
> KUBE_PING in its modules, we have a couple of options how to do it.
>
> Before discussing different implementations, let me quickly go through the
> requirements:
> - We need a configuration stack that works for on-prem and cloud
> deployments with OpenShift as our primary target.
>

OpenShift is not our primary target in community. We should have support
for standalone Docker, Kubernetes and OpenShift. For GCP and EC2 we should
have something that works, but will probably not be able to test this
ourselves.


> - The configuration should be automatic (if it's possible). E.g. if we
> discover that Keycloak is running in the container, we should use proper
> discovery protocol.
> - There needs to be a way to override the discovery protocol manually.
>
> With those requirements in mind, we have a couple of implementation options
> on the table:
> 1. Add more stacks to the configuration, e.g. openshift, azure or gcp. Then
> we use the standard `-Djboss.default.jgroups.stack=<stack>` configuration
> switch.
> 2. Provide more standalone-*.xml configuration files, e.g.
> standalone-ha.xml (for on-prem) or standalone-cloud.xml.

3. Add protocols dynamically using CLI. A similar approach to what we did
> for the Data Grid Cache Service [4].
> 4. Use MULTI_PING protocols [5][6], with multiple discovery protocols on
> the same stack. This will include MPING (for multicasting), KUBE_PING (if
> we can access Kubernetes API), DNS_PING (if Pods are governed by a
> Service).
>

All config should be done through CLI commands. Did you look at how we do
this for DBs? The standalone-*.xml file is modified at runtime to switch
between selected DBs.

To keep things consistent your options are:

* A single CLI script that configures something that works in all scenarios
* A single CLI script that configures alternatives stacks that are enabled
depending on some environment variables
* The same approach as done for DBs where we have different directories for
each DB with different CLI scripts for each DB. This DB specific CLI is
then executed at startup time to re-configure standalone.xml

Anything else will be a maintenance headache.


>
> Option #1 and #2 is somewhat similar to what we did for Infinispan [7]. It
> works quite well but the configuration grows quite quickly and most of the
> protocols (apart from discovery) are duplicated. On the other hand, having
> separate configuration pieces for each use case is very flexible. Having in
> mind that AWS cuts TCP connections, using FD_SOCK might lead to false
> suspicions but on GCP for the instance, FD_SOCK works quite nicely. The CLI
> option (#3), is also very flexible and probably should be implemented only
> in our Docker image. This somehow follows the convention we already started
> with different CLI files for different DBs [8]. Option #4 is brand new
> (implemented in JGroups 4.0.8; we have 4.0.11 as you probably recall). It
> has been specifically designed for this kind of use cases where we want to
> gather discovery data from multiple places. Using this way, we should end
> up with two stacks in standalone-ha.xml file - UDP and TCP.
>
> I honestly need to say, that my heart goes for options #4. However, as far
> as I know it hasn't been battle tested and we might get some surprises. All
> other options are not as elegant as option #4 but they are used somewhere
> in other projects. They are much safer options but they will add some
> maintenance burden on our shoulders.
>

I do like the idea of option #4, but worried about it not working and
causing strange behaviour.


>
> What would you suggest guys? What do you think about all this? @Rado,
> @Paul, @Tristan - Do you have any plans regarding this piece in Wildfly or
> Infinispan?
>
> Thanks,
> Sebastian
>
> [1] https://github.com/jboss-dockerfiles/keycloak/pull/96
> [2] https://github.com/jboss-dockerfiles/keycloak/pull/100
> [3] https://github.com/jboss-dockerfiles/keycloak/pull/116
> [4]
>
> https://github.com/jboss-container-images/datagrid-7-image/blob/datagrid-services-dev/modules/os-datagrid-online-services-configuration/src/main/bash/profiles/caching-service.cli#L37
> [5] http://www.jgroups.org/manual4/index.html#_multi_ping
> [6] https://issues.jboss.org/browse/JGRP-2224
> [7]
>
> https://github.com/infinispan/infinispan/tree/master/server/integration/jgroups/src/main/resources/subsystem-templates
> [8]
>
> https://github.com/jboss-dockerfiles/keycloak/tree/master/server/tools/cli/databases
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From sthorger at redhat.com  Thu Sep 13 04:21:40 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Thu, 13 Sep 2018 10:21:40 +0200
Subject: [keycloak-dev] Clustering configuration
In-Reply-To: <CAA9R9O72q6ho-pdDLx4pk1ONTY1S1oSmQAYkoYJZRRH5p=Oyag@mail.gmail.com>
References: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>
	<CAF5qqoMqsjGbe1ZDeU4wy=qt_kY7=wQW1X7XbtMiXE-Q2bEeKA@mail.gmail.com>
	<bc68249e-fb04-026c-b002-2a91f6937ad1@redhat.com>
	<CAA9R9O72q6ho-pdDLx4pk1ONTY1S1oSmQAYkoYJZRRH5p=Oyag@mail.gmail.com>
Message-ID: <CAJgngAd1be+iCyXdBZ90oUX1Cy9GHEq9Uu4keLEbagn_wjSi_w@mail.gmail.com>

On Thu, 13 Sep 2018 at 09:38, Sebastian Laskawiec <slaskawi at redhat.com>
wrote:

> Answering a several email in a row. Please forgive me for not responding
> each one individually.
>
> Keycloak is actually a web app (rather than a separate app as Infiispan,
> which is more forked of Wildfly rather than layered). We already use a
> database that needs to be somewhere there (either embedded H2, MySQL or
> whatever). Moreover, we also define our data source - KeycloakDS. I also
> expect Keycloak clusters to be rather small, just 2-3 servers for HA.
>
> Having all this in mind, using JDBC_PING might not be a bad idea, I think.
> I guess we are lucky since we can assume that we already have an SQL
> database running. @Bela Ban <bban at redhat.com> I've also seen you fixed
> the crashed members issue mentioned in [1] in 4.0.10. We're on 4.0.11, so
> we should also be good to go.
> However there are a few limitations that are worth mentioning and I would
> like to request some comments from @Marek Posolda <mposolda at redhat.com>, @Stian
> Thorgersen <stian at redhat.com> and @Hynek Mlnarik <hmlnarik at redhat.com>:
> - This makes our dependency to the database even stronger. If we ever
> decided to change the storage, this issue will get back to us like a
> boomerang.
> - When considering cross site replication (with database replication
> turned on), I guess we will need a separate, not-replicated table for
> discovery per each site. Otherwise, sites might override each other by
> writing into the same row in the same table.
> - We have a few data base implementation that we can work with (H2, MySQL,
> PostgreSQL and others but that requires additional drivers). The JDBC_PING
> uses SQL queries so we would need to write them in such a way, that they
> always work regardless to the DB implementation. Since this is at the
> configuration level, we can not use JPA. Those will need to be hand-crafted
> queries. But I guess it shouldn't be a big deal.
>

I'm not keen on JDBC_PING. I know it simple and works, but testing it on
all supported databases will be difficult. Then if we do decide to get rid
of the DB as a strong requirement it would come back to haunt us, but I
think that's something we could address if/when.

When we're talking about testing. We won't be able to do exhaustive testing
here on Docker, OpenShift, Kubernetes, GCE, EC2, ... So we would have to
rely on the community to test and provide us with feedback.


>
> The MULTI_PING also seems very interesting. Thanks a lot for all the
> useful info! @Paul Ferraro <paul.ferraro at redhat.com>, could you please
> tell me when do you plan to implement the multi discovery protocol behavior
> you mentioned in your email? In which Wildfly version this feature will be
> available?
>
> Thanks,
> Sebastian
>
> PS - If you guys spot any problems with JDBC_PING and current version of
> Keycloak, please create a JIRA for us -
> https://issues.jboss.org/browse/KEYCLOAK
>
> [1] https://issues.jboss.org/browse/JGRP-2245
>
> On Wed, Sep 12, 2018 at 5:10 PM Bela Ban <bban at redhat.com> wrote:
>
>>
>>
>> On 12/09/18 14:46, Paul Ferraro wrote:
>> > Hi Sebastian,
>> >
>> > We've planned to support MULTI_PING in WF by adding it to the stack
>> > automatically in the event that a stack contains multiple discovery
>> > protocols [1].  We want to keep this hidden from users, since JGroups
>> > plans to support multiple discovery protocols transparently [2].
>>
>> As I said before, I don't like to add hidden stuff, or else
>> configuration != runtime. However, in this specific case, I agree that
>> this may be a good interim solution until we have multiple discovery
>> protocols without the need for MULTI_PING.
>>
>> > Since we haven't had time to do this yet, we have not yet performed
>> > any testing of MULTI_PING, nor do I know of any users already using
>> > this.
>>
>> Correct. I haven't heard of any users, either.
>>
>> Next steps wrt discovery are me thinking about multiple discovery
>> protocols without MULTI_PING and perhaps a more reactive design
>> (FIND_MBRS_ASYNC) involving CompletableFutures, as suggested by Dan.
>>
>> > If you don't have the bandwidth to toy with MULTI_PING, I would
>> > strongly encourage you to use option #3.  Configuration XML is
>> > permitted to change dramatically between schema versions, and can be a
>> > nightmare for XML manipulation scripts, while the EAP/WF management
>> > CLI is effectively a stable API, where backwards compatibility (for a
>> > specific version range of the model) is a requirement.  Rado recently
>> > refactored our testsuite to uses CLI scripts for modifying
>> > configuration (instead of XML manipulation) and it has considerably
>> > reduced the complexity of our testsuite.  I can't imagine ever going
>> > back.
>> >
>> > CLI scripts for WF/EAP's jgroups subsystem would look considerably
>> > simpler than the scripts you posted above (for the datagrid-jgroups
>> > subsystem).  Specifically:
>> > 1. Rather than remove and redefine an entire protocol stack, you can
>> > remove a specific protocol and insert an new one in the stack at a
>> > specific index.
>> > 2. pbcast.NAKACK2's use_mcast_xmit attribute is auto-disabled when the
>> > transport does not support multicast, so this property never needs to
>> > be set explicitly.
>> > 3. All clustering operations support the
>> > {allow-resource-service-restart=true} operation header to avoid a full
>> > server reload by restarting only affected services.  Batching
>> > (especially when containing remove/add operation pairs) is encouraged
>> > to eliminate redundant service restarts.
>> >
>> > [1] https://issues.jboss.org/browse/WFLY-9723
>> > [2] https://issues.jboss.org/browse/JGRP-2230
>> > On Wed, Sep 12, 2018 at 4:00 AM Sebastian Laskawiec <
>> slaskawi at redhat.com> wrote:
>> >>
>> >> Hey guys,
>> >>
>> >> During our weekly sync meeting, Stian asked me to look into different
>> options for clustering in Keycloak server. This topic has quite hot with
>> the context of our Docker image (see the proposed community contributions
>> [1][2][3]). Since we are based on WF 13, which uses JGroups 4.0.11 and has
>> KUBE_PING in its modules, we have a couple of options how to do it.
>> >>
>> >> Before discussing different implementations, let me quickly go through
>> the requirements:
>> >> - We need a configuration stack that works for on-prem and cloud
>> deployments with OpenShift as our primary target.
>> >> - The configuration should be automatic (if it's possible). E.g. if we
>> discover that Keycloak is running in the container, we should use proper
>> discovery protocol.
>> >> - There needs to be a way to override the discovery protocol manually.
>> >>
>> >> With those requirements in mind, we have a couple of implementation
>> options on the table:
>> >> 1. Add more stacks to the configuration, e.g. openshift, azure or gcp.
>> Then we use the standard `-Djboss.default.jgroups.stack=<stack>`
>> configuration switch.
>> >> 2. Provide more standalone-*.xml configuration files, e.g.
>> standalone-ha.xml (for on-prem) or standalone-cloud.xml.
>> >> 3. Add protocols dynamically using CLI. A similar approach to what we
>> did for the Data Grid Cache Service [4].
>> >> 4. Use MULTI_PING protocols [5][6], with multiple discovery protocols
>> on the same stack. This will include MPING (for multicasting), KUBE_PING
>> (if we can access Kubernetes API), DNS_PING (if Pods are governed by a
>> Service).
>> >>
>> >> Option #1 and #2 is somewhat similar to what we did for Infinispan
>> [7]. It works quite well but the configuration grows quite quickly and most
>> of the protocols (apart from discovery) are duplicated. On the other hand,
>> having separate configuration pieces for each use case is very flexible.
>> Having in mind that AWS cuts TCP connections, using FD_SOCK might lead to
>> false suspicions but on GCP for the instance, FD_SOCK works quite nicely.
>> The CLI option (#3), is also very flexible and probably should be
>> implemented only in our Docker image. This somehow follows the convention
>> we already started with different CLI files for different DBs [8]. Option
>> #4 is brand new (implemented in JGroups 4.0.8; we have 4.0.11 as you
>> probably recall). It has been specifically designed for this kind of use
>> cases where we want to gather discovery data from multiple places. Using
>> this way, we should end up with two stacks in standalone-ha.xml file - UDP
>> and TCP.
>> >>
>> >> I honestly need to say, that my heart goes for options #4. However, as
>> far as I know it hasn't been battle tested and we might get some surprises.
>> All other options are not as elegant as option #4 but they are used
>> somewhere in other projects. They are much safer options but they will add
>> some maintenance burden on our shoulders.
>> >>
>> >> What would you suggest guys? What do you think about all this? @Rado,
>> @Paul, @Tristan - Do you have any plans regarding this piece in Wildfly or
>> Infinispan?
>> >>
>> >> Thanks,
>> >> Sebastian
>> >>
>> >> [1] https://github.com/jboss-dockerfiles/keycloak/pull/96
>> >> [2] https://github.com/jboss-dockerfiles/keycloak/pull/100
>> >> [3] https://github.com/jboss-dockerfiles/keycloak/pull/116
>> >> [4]
>> https://github.com/jboss-container-images/datagrid-7-image/blob/datagrid-services-dev/modules/os-datagrid-online-services-configuration/src/main/bash/profiles/caching-service.cli#L37
>> >> [5] http://www.jgroups.org/manual4/index.html#_multi_ping
>> >> [6] https://issues.jboss.org/browse/JGRP-2224
>> >> [7]
>> https://github.com/infinispan/infinispan/tree/master/server/integration/jgroups/src/main/resources/subsystem-templates
>> >> [8]
>> https://github.com/jboss-dockerfiles/keycloak/tree/master/server/tools/cli/databases
>>
>> --
>> Bela Ban | http://www.jgroups.org
>>
>>

From sthorger at redhat.com  Thu Sep 13 05:24:19 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Thu, 13 Sep 2018 11:24:19 +0200
Subject: [keycloak-dev] Removal of user-storage-jpa and
	user-storage-simple
In-Reply-To: <CAM5SUC5-utpxtUxEneQbs8Fvzpfx7ortTg317eVTm8MrdJEuLg@mail.gmail.com>
References: <CAM5SUC5-utpxtUxEneQbs8Fvzpfx7ortTg317eVTm8MrdJEuLg@mail.gmail.com>
Message-ID: <CAJgngAeVKAbmAbXY9y549UGDBge8SzNRn5dbdFrXmKvqk6_Qdg@mail.gmail.com>

Go for it

On Thu, 13 Sep 2018 at 00:09, Bruno Oliveira <bruno at abstractj.org> wrote:

> Good night,
>
> Now that we have user-storage-jpa and user-storage-simple examples
> into the Quickstarts repository
> (https://github.com/keycloak/keycloak-quickstarts). I would like to
> remove these projects from Keycloak repository
> (https://github.com/keycloak/keycloak/tree/master/examples/providers).
>
> Any concerns about this?
>
> --
> - abstractj
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From slaskawi at redhat.com  Thu Sep 13 08:50:15 2018
From: slaskawi at redhat.com (Sebastian Laskawiec)
Date: Thu, 13 Sep 2018 14:50:15 +0200
Subject: [keycloak-dev] Clustering configuration
In-Reply-To: <CAJgngAd1be+iCyXdBZ90oUX1Cy9GHEq9Uu4keLEbagn_wjSi_w@mail.gmail.com>
References: <CAA9R9O6pgNhoo=Ps1V8SXWQBH1pyX+T4rce-HJy-s7M4iBZVBQ@mail.gmail.com>
	<CAF5qqoMqsjGbe1ZDeU4wy=qt_kY7=wQW1X7XbtMiXE-Q2bEeKA@mail.gmail.com>
	<bc68249e-fb04-026c-b002-2a91f6937ad1@redhat.com>
	<CAA9R9O72q6ho-pdDLx4pk1ONTY1S1oSmQAYkoYJZRRH5p=Oyag@mail.gmail.com>
	<CAJgngAd1be+iCyXdBZ90oUX1Cy9GHEq9Uu4keLEbagn_wjSi_w@mail.gmail.com>
Message-ID: <CAA9R9O6wBinXrOsJpySCCFHhP6t=e-6-DGB1nfuZif3s38Fo1A@mail.gmail.com>

Sorry something got wrong and I forgot to include the dev mailing list.

We've just had a discussion on this topic with @Stian Thorgersen
<stian at redhat.com>. Here's what we agreed upon:
- We want to have an extensible mechanism for adding new discovery
protocols for different environments. We plan to implement something
similar to what we have for DBs.
- We will be able to switch into different configuration options using
environment variables.
- For the OpenShift use case, we'll focus on DNS_PING.
- Later on, we'll ask for community help especially around JDBC_PING.

Let me look at the code and implement a proposal. I'll ask you guys for a
review once I have a working PR.

Thanks again to all of you for a very useful info!

On Thu, Sep 13, 2018 at 10:21 AM Stian Thorgersen <sthorger at redhat.com>
wrote:

>
>
> On Thu, 13 Sep 2018 at 09:38, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
>
>> Answering a several email in a row. Please forgive me for not responding
>> each one individually.
>>
>> Keycloak is actually a web app (rather than a separate app as Infiispan,
>> which is more forked of Wildfly rather than layered). We already use a
>> database that needs to be somewhere there (either embedded H2, MySQL or
>> whatever). Moreover, we also define our data source - KeycloakDS. I also
>> expect Keycloak clusters to be rather small, just 2-3 servers for HA.
>>
>> Having all this in mind, using JDBC_PING might not be a bad idea, I
>> think. I guess we are lucky since we can assume that we already have an SQL
>> database running. @Bela Ban <bban at redhat.com> I've also seen you fixed
>> the crashed members issue mentioned in [1] in 4.0.10. We're on 4.0.11, so
>> we should also be good to go.
>> However there are a few limitations that are worth mentioning and I would
>> like to request some comments from @Marek Posolda <mposolda at redhat.com>, @Stian
>> Thorgersen <stian at redhat.com> and @Hynek Mlnarik <hmlnarik at redhat.com>:
>> - This makes our dependency to the database even stronger. If we ever
>> decided to change the storage, this issue will get back to us like a
>> boomerang.
>> - When considering cross site replication (with database replication
>> turned on), I guess we will need a separate, not-replicated table for
>> discovery per each site. Otherwise, sites might override each other by
>> writing into the same row in the same table.
>> - We have a few data base implementation that we can work with (H2,
>> MySQL, PostgreSQL and others but that requires additional drivers). The
>> JDBC_PING uses SQL queries so we would need to write them in such a way,
>> that they always work regardless to the DB implementation. Since this is at
>> the configuration level, we can not use JPA. Those will need to be
>> hand-crafted queries. But I guess it shouldn't be a big deal.
>>
>
> I'm not keen on JDBC_PING. I know it simple and works, but testing it on
> all supported databases will be difficult. Then if we do decide to get rid
> of the DB as a strong requirement it would come back to haunt us, but I
> think that's something we could address if/when.
>
> When we're talking about testing. We won't be able to do exhaustive
> testing here on Docker, OpenShift, Kubernetes, GCE, EC2, ... So we would
> have to rely on the community to test and provide us with feedback.
>
>
>>
>> The MULTI_PING also seems very interesting. Thanks a lot for all the
>> useful info! @Paul Ferraro <paul.ferraro at redhat.com>, could you please
>> tell me when do you plan to implement the multi discovery protocol behavior
>> you mentioned in your email? In which Wildfly version this feature will be
>> available?
>>
>> Thanks,
>> Sebastian
>>
>> PS - If you guys spot any problems with JDBC_PING and current version of
>> Keycloak, please create a JIRA for us -
>> https://issues.jboss.org/browse/KEYCLOAK
>>
>> [1] https://issues.jboss.org/browse/JGRP-2245
>>
>> On Wed, Sep 12, 2018 at 5:10 PM Bela Ban <bban at redhat.com> wrote:
>>
>>>
>>>
>>> On 12/09/18 14:46, Paul Ferraro wrote:
>>> > Hi Sebastian,
>>> >
>>> > We've planned to support MULTI_PING in WF by adding it to the stack
>>> > automatically in the event that a stack contains multiple discovery
>>> > protocols [1].  We want to keep this hidden from users, since JGroups
>>> > plans to support multiple discovery protocols transparently [2].
>>>
>>> As I said before, I don't like to add hidden stuff, or else
>>> configuration != runtime. However, in this specific case, I agree that
>>> this may be a good interim solution until we have multiple discovery
>>> protocols without the need for MULTI_PING.
>>>
>>> > Since we haven't had time to do this yet, we have not yet performed
>>> > any testing of MULTI_PING, nor do I know of any users already using
>>> > this.
>>>
>>> Correct. I haven't heard of any users, either.
>>>
>>> Next steps wrt discovery are me thinking about multiple discovery
>>> protocols without MULTI_PING and perhaps a more reactive design
>>> (FIND_MBRS_ASYNC) involving CompletableFutures, as suggested by Dan.
>>>
>>> > If you don't have the bandwidth to toy with MULTI_PING, I would
>>> > strongly encourage you to use option #3.  Configuration XML is
>>> > permitted to change dramatically between schema versions, and can be a
>>> > nightmare for XML manipulation scripts, while the EAP/WF management
>>> > CLI is effectively a stable API, where backwards compatibility (for a
>>> > specific version range of the model) is a requirement.  Rado recently
>>> > refactored our testsuite to uses CLI scripts for modifying
>>> > configuration (instead of XML manipulation) and it has considerably
>>> > reduced the complexity of our testsuite.  I can't imagine ever going
>>> > back.
>>> >
>>> > CLI scripts for WF/EAP's jgroups subsystem would look considerably
>>> > simpler than the scripts you posted above (for the datagrid-jgroups
>>> > subsystem).  Specifically:
>>> > 1. Rather than remove and redefine an entire protocol stack, you can
>>> > remove a specific protocol and insert an new one in the stack at a
>>> > specific index.
>>> > 2. pbcast.NAKACK2's use_mcast_xmit attribute is auto-disabled when the
>>> > transport does not support multicast, so this property never needs to
>>> > be set explicitly.
>>> > 3. All clustering operations support the
>>> > {allow-resource-service-restart=true} operation header to avoid a full
>>> > server reload by restarting only affected services.  Batching
>>> > (especially when containing remove/add operation pairs) is encouraged
>>> > to eliminate redundant service restarts.
>>> >
>>> > [1] https://issues.jboss.org/browse/WFLY-9723
>>> > [2] https://issues.jboss.org/browse/JGRP-2230
>>> > On Wed, Sep 12, 2018 at 4:00 AM Sebastian Laskawiec <
>>> slaskawi at redhat.com> wrote:
>>> >>
>>> >> Hey guys,
>>> >>
>>> >> During our weekly sync meeting, Stian asked me to look into different
>>> options for clustering in Keycloak server. This topic has quite hot with
>>> the context of our Docker image (see the proposed community contributions
>>> [1][2][3]). Since we are based on WF 13, which uses JGroups 4.0.11 and has
>>> KUBE_PING in its modules, we have a couple of options how to do it.
>>> >>
>>> >> Before discussing different implementations, let me quickly go
>>> through the requirements:
>>> >> - We need a configuration stack that works for on-prem and cloud
>>> deployments with OpenShift as our primary target.
>>> >> - The configuration should be automatic (if it's possible). E.g. if
>>> we discover that Keycloak is running in the container, we should use proper
>>> discovery protocol.
>>> >> - There needs to be a way to override the discovery protocol manually.
>>> >>
>>> >> With those requirements in mind, we have a couple of implementation
>>> options on the table:
>>> >> 1. Add more stacks to the configuration, e.g. openshift, azure or
>>> gcp. Then we use the standard `-Djboss.default.jgroups.stack=<stack>`
>>> configuration switch.
>>> >> 2. Provide more standalone-*.xml configuration files, e.g.
>>> standalone-ha.xml (for on-prem) or standalone-cloud.xml.
>>> >> 3. Add protocols dynamically using CLI. A similar approach to what we
>>> did for the Data Grid Cache Service [4].
>>> >> 4. Use MULTI_PING protocols [5][6], with multiple discovery protocols
>>> on the same stack. This will include MPING (for multicasting), KUBE_PING
>>> (if we can access Kubernetes API), DNS_PING (if Pods are governed by a
>>> Service).
>>> >>
>>> >> Option #1 and #2 is somewhat similar to what we did for Infinispan
>>> [7]. It works quite well but the configuration grows quite quickly and most
>>> of the protocols (apart from discovery) are duplicated. On the other hand,
>>> having separate configuration pieces for each use case is very flexible.
>>> Having in mind that AWS cuts TCP connections, using FD_SOCK might lead to
>>> false suspicions but on GCP for the instance, FD_SOCK works quite nicely.
>>> The CLI option (#3), is also very flexible and probably should be
>>> implemented only in our Docker image. This somehow follows the convention
>>> we already started with different CLI files for different DBs [8]. Option
>>> #4 is brand new (implemented in JGroups 4.0.8; we have 4.0.11 as you
>>> probably recall). It has been specifically designed for this kind of use
>>> cases where we want to gather discovery data from multiple places. Using
>>> this way, we should end up with two stacks in standalone-ha.xml file - UDP
>>> and TCP.
>>> >>
>>> >> I honestly need to say, that my heart goes for options #4. However,
>>> as far as I know it hasn't been battle tested and we might get some
>>> surprises. All other options are not as elegant as option #4 but they are
>>> used somewhere in other projects. They are much safer options but they will
>>> add some maintenance burden on our shoulders.
>>> >>
>>> >> What would you suggest guys? What do you think about all this? @Rado,
>>> @Paul, @Tristan - Do you have any plans regarding this piece in Wildfly or
>>> Infinispan?
>>> >>
>>> >> Thanks,
>>> >> Sebastian
>>> >>
>>> >> [1] https://github.com/jboss-dockerfiles/keycloak/pull/96
>>> >> [2] https://github.com/jboss-dockerfiles/keycloak/pull/100
>>> >> [3] https://github.com/jboss-dockerfiles/keycloak/pull/116
>>> >> [4]
>>> https://github.com/jboss-container-images/datagrid-7-image/blob/datagrid-services-dev/modules/os-datagrid-online-services-configuration/src/main/bash/profiles/caching-service.cli#L37
>>> >> [5] http://www.jgroups.org/manual4/index.html#_multi_ping
>>> >> [6] https://issues.jboss.org/browse/JGRP-2224
>>> >> [7]
>>> https://github.com/infinispan/infinispan/tree/master/server/integration/jgroups/src/main/resources/subsystem-templates
>>> >> [8]
>>> https://github.com/jboss-dockerfiles/keycloak/tree/master/server/tools/cli/databases
>>>
>>> --
>>> Bela Ban | http://www.jgroups.org
>>>
>>>

From alkhandani at gmail.com  Thu Sep 13 10:50:21 2018
From: alkhandani at gmail.com (Jeus Geek)
Date: Thu, 13 Sep 2018 19:20:21 +0430
Subject: [keycloak-dev] Invalid CORS request
Message-ID: <CA+g8j=NXE0vGJJH8fpG_cMzCPBooYQc2uG=EeVAodiAegpa0rQ@mail.gmail.com>

Hi
I add a new tab (SMS) for configuration service.
this service has 2 <key, value> same for a register in the database
I used from SMTP (Email) tab to configuration develop that. when trying to
set configuration at system get an error
"19:06:11,581 DEBUG [org.keycloak.services.resources.Cors] (default task-7)
Invalid CORS request: origin http://127.0.0.1:8080 not in allowed origins
[]"
I can't find anywhere that defined CORSs for new realm tabs.

Thanks
Jeus

From Sebastian.Schuster at bosch-si.com  Thu Sep 13 11:22:00 2018
From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST-CSS/BSV-OS))
Date: Thu, 13 Sep 2018 15:22:00 +0000
Subject: [keycloak-dev] Clustering configuration
In-Reply-To: <6ffb-5b98f980-7-33393680@14777610>
References: <a8a8ada680c24b6e910d2b8bf8c3b07a@bosch-si.com>
	<6ffb-5b98f980-7-33393680@14777610>
Message-ID: <0afedb06d5444a00bbc94a197719d2a8@bosch-si.com>

Hi C?dric,

Thanks for your help. I tested your configuration and it seems to work but the effects are still kind of unexpected.

It looks like the cluster is formed correctly. But the way JDBC_PING itself works is completely different from before. In the past, every node removed and readded itself to the JGROUPSPING table every few seconds. Now it looks like only the master node updates the JGROUPSPING table and only when it detects a change. It also always inserts its own address for every node. That makes the JGROUPSPING table look kind of strange as it does not contain the correct addresses for all the nodes. I have not found documentation for such a big change so I am not sure this is all intended...

Best regards,
Sebastian

Mit freundlichen Gr??en / Best regards

Dr.-Ing.  Sebastian Schuster

Open Source Services (INST-CSS/BSV-OS) 
Bosch?Software Innovations?GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B 
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn 




-----Original Message-----
From: cedric at couralet.eu <cedric at couralet.eu> 
Sent: Mittwoch, 12. September 2018 13:33
To: Schuster Sebastian (INST-CSS/BSV-OS) <Sebastian.Schuster at bosch-si.com>
Cc: Thomas Darimont <thomas.darimont at googlemail.com>; Radoslav Husar <rhusar at redhat.com>; Bela Ban <bban at redhat.com>; Paul Ferraro <paul.ferraro at redhat.com>; keycloak-dev <keycloak-dev at lists.jboss.org>; Tarrant, Tristan <ttarrant at redhat.com>
Subject: Re: [keycloak-dev] Clustering configuration

Hi Sebastian,
 
Le Mercredi, Septembre 12, 2018 11:56 CEST, "Schuster Sebastian (INST-CSS/BSV-OS)" <Sebastian.Schuster at bosch-si.com> a ?crit: 
 
> Guess what, our JDBC_PING configuration not working with 4.4.0.Final is what I am currently working on.

If it helps, I had some difficulties configuring JDBC_PING on keycloak 4.4.0.Final. 

My final working configuration is :
       <subsystem xmlns="urn:jboss:domain:jgroups:6.0">
            <channels default="ee">
                <channel name="ee" stack="tcpping"/>
            </channels>
            <stacks>
                <stack name="tcpping">
                    <transport type="TCP" socket-binding="jgroups-tcp">
                        <property name="external_addr">
                            ${jgroups.bind.address:127.0.0.1}
                        </property>
                        <property name="bind_addr">
                            ${jgroups.bind_addr:SITE_LOCAL}
                        </property>
                    </transport>
                    <jdbc-protocol type="JDBC_PING" data-source="KeycloakDS">
                        <property name="initialize_sql">
                            CREATE TABLE IF NOT EXISTS JGROUPSPING (own_addr varchar(200) NOT NULL,bind_addr varchar(200) NOT NULL,created timestamp NOT NULL,cluster_name varchar(200) NOT NULL,ping_data BYTEA,constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name))
                        </property>
                        <property name="insert_single_sql">
                            INSERT INTO JGROUPSPING (own_addr, bind_addr, created, cluster_name, ping_data) values (?,'${jgroups.bind.address:127.0.0.1}',NOW(), ?, ?)
                        </property>
                        <property name="delete_single_sql">
                            DELETE FROM JGROUPSPING WHERE own_addr=? AND cluster_name=?
                        </property>
                        <property name="select_all_pingdata_sql">
                            SELECT ping_data FROM JGROUPSPING WHERE cluster_name=?;
                        </property>
                    </jdbc-protocol>
                    <protocol type="MERGE3"/>
                    <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd">
                        <property name="external_addr">
                            ${jgroups.bind.address:127.0.0.1}
                        </property>
                    </protocol>
                    <protocol type="FD"/>
                    <protocol type="VERIFY_SUSPECT"/>
                    <protocol type="pbcast.NAKACK2"/>
                    <protocol type="UNICAST3"/>
                    <protocol type="pbcast.STABLE"/>
                    <protocol type="pbcast.GMS"/>
                    <protocol type="MFC"/>
                    <protocol type="FRAG2"/>
                </stack>
            </stacks>
        </subsystem>


I do it in my own docker image where I change the configuration with jboss-cli (as in the officilal) with this file :
/subsystem=infinispan/cache-container=keycloak/distributed-cache=sessions:remove
/subsystem=infinispan/cache-container=keycloak/replicated-cache=sessions:add()
/subsystem=infinispan/cache-container=keycloak/replicated-cache=sessions:write-attribute(name="mode",value="SYNC")

/subsystem=infinispan/cache-container=keycloak/distributed-cache=authenticationSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=clientSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineClientSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures:remove

/subsystem=infinispan/cache-container=keycloak/distributed-cache=authenticationSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=clientSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineClientSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures:add(mode="SYNC",owners="2")

/subsystem=jgroups/stack=tcpping:add()
/subsystem=jgroups/stack=tcpping/transport=TCP:add(socket-binding=jgroups-tcp)
/subsystem=jgroups/stack=tcpping/transport=TCP/property=external_addr:add(value=${jgroups.bind.address:127.0.0.1})
/subsystem=jgroups/stack=tcpping/transport=TCP/property=bind_addr:add(value=${jgroups.bind_addr:SITE_LOCAL})


/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING:add(data-source="KeycloakDS", properties=[initialize_sql="CREATE TABLE IF NOT EXISTS JGROUPSPING (own_addr varchar(200) NOT NULL,bind_addr varchar(200) NOT NULL,created timestamp NOT NULL,cluster_name varchar(200) NOT NULL,ping_data BYTEA,constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name))",insert_single_sql="INSERT INTO JGROUPSPING (own_addr, bind_addr, created, cluster_name, ping_data) values (?,'${jgroups.bind.address:127.0.0.1}',NOW(), ?, ?)",delete_single_sql="DELETE FROM JGROUPSPING WHERE own_addr=? AND cluster_name=?",select_all_pingdata_sql="SELECT ping_data FROM JGROUPSPING WHERE cluster_name=?;"])

/subsystem=jgroups/stack=tcpping/protocol=MERGE3:add()
/subsystem=jgroups/stack=tcpping:add-protocol(type="FD_SOCK",socket-binding="jgroups-tcp-fd")
/subsystem=jgroups/stack=tcpping/protocol=FD_SOCK/property=external_addr:add(value=${jgroups.bind.address:127.0.0.1})
/subsystem=jgroups/stack=tcpping/protocol=FD:add()
/subsystem=jgroups/stack=tcpping/protocol=VERIFY_SUSPECT:add()
/subsystem=jgroups/stack=tcpping/protocol=pbcast.NAKACK2:add()
/subsystem=jgroups/stack=tcpping/protocol=UNICAST3:add()
/subsystem=jgroups/stack=tcpping/protocol=pbcast.STABLE:add()
/subsystem=jgroups/stack=tcpping/protocol=pbcast.GMS:add()
/subsystem=jgroups/stack=tcpping/protocol=MFC:add()
/subsystem=jgroups/stack=tcpping/protocol=FRAG2:add()

/subsystem=jgroups/channel=ee:remove
/subsystem=jgroups/channel=ee:add(stack=tcpping)
/subsystem=jgroups:write-attribute(name=default-channel, value=ee)

/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp:write-attribute(name="interface",value="private")
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd:add()
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd:write-attribute(name="interface",value="private")
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd:write-attribute(name="port",value="57600")


/subsystem=jgroups/stack=tcp:remove
/subsystem=jgroups/stack=udp:remove


I am not sure all is good in this but it works (in my environment :) ).

(the difficulties I had was that if the cli file is like :
...
	/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING:add(data-source="KeycloakDS")
	/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING/property=datasource_jndi_name:add(value=java:jboss/datasources/KeycloakDS)
/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING/property=otherproperty:add(value=other_value)
...

the configuration in xml is :
  <protocol type="org.apache.jgroups.JDBC_PING" >

Which doesn't work (don't know why). There is not a lot of documentation on this, so I'm listening to all suggestions.

Cheers,
C?dric Couralet



From Sebastian.Schuster at bosch-si.com  Thu Sep 13 11:30:22 2018
From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST-CSS/BSV-OS))
Date: Thu, 13 Sep 2018 15:30:22 +0000
Subject: [keycloak-dev] Clustering configuration
In-Reply-To: <6ffb-5b98f980-7-33393680@14777610>
References: <a8a8ada680c24b6e910d2b8bf8c3b07a@bosch-si.com>
	<6ffb-5b98f980-7-33393680@14777610>
Message-ID: <3c7cf10ce6fe4ccca92cd7fad75a6f25@bosch-si.com>

Btw. the fact that there is only one IP address in JGROUPSPING is of course due to the configured insert_single_sql statement always taking the address of the local node. This will always be the master's address if it is the only one writing to the table...

Best regards,
Sebastian

Mit freundlichen Gr??en / Best regards

Dr.-Ing.  Sebastian Schuster

Open Source Services (INST-CSS/BSV-OS) 
Bosch?Software Innovations?GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B 
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn 




-----Original Message-----
From: cedric at couralet.eu <cedric at couralet.eu> 
Sent: Mittwoch, 12. September 2018 13:33
To: Schuster Sebastian (INST-CSS/BSV-OS) <Sebastian.Schuster at bosch-si.com>
Cc: Thomas Darimont <thomas.darimont at googlemail.com>; Radoslav Husar <rhusar at redhat.com>; Bela Ban <bban at redhat.com>; Paul Ferraro <paul.ferraro at redhat.com>; keycloak-dev <keycloak-dev at lists.jboss.org>; Tarrant, Tristan <ttarrant at redhat.com>
Subject: Re: [keycloak-dev] Clustering configuration

Hi Sebastian,
 
Le Mercredi, Septembre 12, 2018 11:56 CEST, "Schuster Sebastian (INST-CSS/BSV-OS)" <Sebastian.Schuster at bosch-si.com> a ?crit: 
 
> Guess what, our JDBC_PING configuration not working with 4.4.0.Final is what I am currently working on.

If it helps, I had some difficulties configuring JDBC_PING on keycloak 4.4.0.Final. 

My final working configuration is :
       <subsystem xmlns="urn:jboss:domain:jgroups:6.0">
            <channels default="ee">
                <channel name="ee" stack="tcpping"/>
            </channels>
            <stacks>
                <stack name="tcpping">
                    <transport type="TCP" socket-binding="jgroups-tcp">
                        <property name="external_addr">
                            ${jgroups.bind.address:127.0.0.1}
                        </property>
                        <property name="bind_addr">
                            ${jgroups.bind_addr:SITE_LOCAL}
                        </property>
                    </transport>
                    <jdbc-protocol type="JDBC_PING" data-source="KeycloakDS">
                        <property name="initialize_sql">
                            CREATE TABLE IF NOT EXISTS JGROUPSPING (own_addr varchar(200) NOT NULL,bind_addr varchar(200) NOT NULL,created timestamp NOT NULL,cluster_name varchar(200) NOT NULL,ping_data BYTEA,constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name))
                        </property>
                        <property name="insert_single_sql">
                            INSERT INTO JGROUPSPING (own_addr, bind_addr, created, cluster_name, ping_data) values (?,'${jgroups.bind.address:127.0.0.1}',NOW(), ?, ?)
                        </property>
                        <property name="delete_single_sql">
                            DELETE FROM JGROUPSPING WHERE own_addr=? AND cluster_name=?
                        </property>
                        <property name="select_all_pingdata_sql">
                            SELECT ping_data FROM JGROUPSPING WHERE cluster_name=?;
                        </property>
                    </jdbc-protocol>
                    <protocol type="MERGE3"/>
                    <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd">
                        <property name="external_addr">
                            ${jgroups.bind.address:127.0.0.1}
                        </property>
                    </protocol>
                    <protocol type="FD"/>
                    <protocol type="VERIFY_SUSPECT"/>
                    <protocol type="pbcast.NAKACK2"/>
                    <protocol type="UNICAST3"/>
                    <protocol type="pbcast.STABLE"/>
                    <protocol type="pbcast.GMS"/>
                    <protocol type="MFC"/>
                    <protocol type="FRAG2"/>
                </stack>
            </stacks>
        </subsystem>


I do it in my own docker image where I change the configuration with jboss-cli (as in the officilal) with this file :
/subsystem=infinispan/cache-container=keycloak/distributed-cache=sessions:remove
/subsystem=infinispan/cache-container=keycloak/replicated-cache=sessions:add()
/subsystem=infinispan/cache-container=keycloak/replicated-cache=sessions:write-attribute(name="mode",value="SYNC")

/subsystem=infinispan/cache-container=keycloak/distributed-cache=authenticationSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=clientSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineClientSessions:remove
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures:remove

/subsystem=infinispan/cache-container=keycloak/distributed-cache=authenticationSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=clientSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=offlineClientSessions:add(mode="SYNC",owners="2")
/subsystem=infinispan/cache-container=keycloak/distributed-cache=loginFailures:add(mode="SYNC",owners="2")

/subsystem=jgroups/stack=tcpping:add()
/subsystem=jgroups/stack=tcpping/transport=TCP:add(socket-binding=jgroups-tcp)
/subsystem=jgroups/stack=tcpping/transport=TCP/property=external_addr:add(value=${jgroups.bind.address:127.0.0.1})
/subsystem=jgroups/stack=tcpping/transport=TCP/property=bind_addr:add(value=${jgroups.bind_addr:SITE_LOCAL})


/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING:add(data-source="KeycloakDS", properties=[initialize_sql="CREATE TABLE IF NOT EXISTS JGROUPSPING (own_addr varchar(200) NOT NULL,bind_addr varchar(200) NOT NULL,created timestamp NOT NULL,cluster_name varchar(200) NOT NULL,ping_data BYTEA,constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name))",insert_single_sql="INSERT INTO JGROUPSPING (own_addr, bind_addr, created, cluster_name, ping_data) values (?,'${jgroups.bind.address:127.0.0.1}',NOW(), ?, ?)",delete_single_sql="DELETE FROM JGROUPSPING WHERE own_addr=? AND cluster_name=?",select_all_pingdata_sql="SELECT ping_data FROM JGROUPSPING WHERE cluster_name=?;"])

/subsystem=jgroups/stack=tcpping/protocol=MERGE3:add()
/subsystem=jgroups/stack=tcpping:add-protocol(type="FD_SOCK",socket-binding="jgroups-tcp-fd")
/subsystem=jgroups/stack=tcpping/protocol=FD_SOCK/property=external_addr:add(value=${jgroups.bind.address:127.0.0.1})
/subsystem=jgroups/stack=tcpping/protocol=FD:add()
/subsystem=jgroups/stack=tcpping/protocol=VERIFY_SUSPECT:add()
/subsystem=jgroups/stack=tcpping/protocol=pbcast.NAKACK2:add()
/subsystem=jgroups/stack=tcpping/protocol=UNICAST3:add()
/subsystem=jgroups/stack=tcpping/protocol=pbcast.STABLE:add()
/subsystem=jgroups/stack=tcpping/protocol=pbcast.GMS:add()
/subsystem=jgroups/stack=tcpping/protocol=MFC:add()
/subsystem=jgroups/stack=tcpping/protocol=FRAG2:add()

/subsystem=jgroups/channel=ee:remove
/subsystem=jgroups/channel=ee:add(stack=tcpping)
/subsystem=jgroups:write-attribute(name=default-channel, value=ee)

/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp:write-attribute(name="interface",value="private")
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd:add()
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd:write-attribute(name="interface",value="private")
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd:write-attribute(name="port",value="57600")


/subsystem=jgroups/stack=tcp:remove
/subsystem=jgroups/stack=udp:remove


I am not sure all is good in this but it works (in my environment :) ).

(the difficulties I had was that if the cli file is like :
...
	/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING:add(data-source="KeycloakDS")
	/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING/property=datasource_jndi_name:add(value=java:jboss/datasources/KeycloakDS)
/subsystem=jgroups/stack=tcpping/protocol=JDBC_PING/property=otherproperty:add(value=other_value)
...

the configuration in xml is :
  <protocol type="org.apache.jgroups.JDBC_PING" >

Which doesn't work (don't know why). There is not a lot of documentation on this, so I'm listening to all suggestions.

Cheers,
C?dric Couralet



From ssilvert at redhat.com  Fri Sep 14 16:21:18 2018
From: ssilvert at redhat.com (Stan Silvert)
Date: Fri, 14 Sep 2018 16:21:18 -0400
Subject: [keycloak-dev] WTF: Jenkins CI Build failure
Message-ID: <944466e6-d579-e570-4a75-96fa6dc56369@redhat.com>

So does anybody know what's going on with the Jenkins CI build???It says 
that it failed but when I try to click on the Details link it says 
"keycloak-jenkins.com?s server IP address could not be found."

See https://github.com/keycloak/keycloak/pull/5569


From bruno at abstractj.org  Fri Sep 14 17:50:02 2018
From: bruno at abstractj.org (Bruno Oliveira)
Date: Fri, 14 Sep 2018 18:50:02 -0300
Subject: [keycloak-dev] WTF: Jenkins CI Build failure
In-Reply-To: <944466e6-d579-e570-4a75-96fa6dc56369@redhat.com>
References: <944466e6-d579-e570-4a75-96fa6dc56369@redhat.com>
Message-ID: <CAM5SUC5PYf7D-BQm3PQaRXtGve8wkn3s5sjoMxDPBQmVRHi4UQ@mail.gmail.com>

That's odd, even connected into the VPN I can't see the logs. Maybe ping Pavel?
On Fri, Sep 14, 2018 at 6:27 PM Stan Silvert <ssilvert at redhat.com> wrote:
>
> So does anybody know what's going on with the Jenkins CI build???It says
> that it failed but when I try to click on the Details link it says
> "keycloak-jenkins.com?s server IP address could not be found."
>
> See https://github.com/keycloak/keycloak/pull/5569
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev



-- 
- abstractj


From dpalmer at redhat.com  Tue Sep 18 16:02:08 2018
From: dpalmer at redhat.com (Douglas Palmer)
Date: Tue, 18 Sep 2018 13:02:08 -0700
Subject: [keycloak-dev] Device fingerprinting
Message-ID: <58D14DAF-65DE-430C-9ACB-10E285694870@redhat.com>

Hi everyone

I haven?t managed to find an open source solution to device fingerprinting which gives us everything we need. This library however gets us most of the way there http://valve.github.io/fingerprintjs2 <http://valve.github.io/fingerprintjs2>. It doesn?t give us enough information to distinguish between a desktop and a laptop but it will let us correlate devices and we can distinguish between a PC, a tablet and a phone. We can also get the OS, Browser and Versions from the user agent string.

I have taken a look at a few sites the track device sessions. Apple can tell the difference between an iMac, a MacBook, an iPad and an iPhone. Facebook, GitHub, Google, LinkedIn and Pinterest don?t distinguish between an iMac and a MacBook. So maybe the library above is enough.

I also came across the following article from the EFF which casts doubt on the legality of digital fingerprinting in Europe. https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-? <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers>

Does anyone have any input on any of this? Is there a better library that I have missed? Should we stick to parsing the user agent to avoid potential problems with GDPR?

Regards
Doug

From sthorger at redhat.com  Wed Sep 19 13:18:01 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Wed, 19 Sep 2018 19:18:01 +0200
Subject: [keycloak-dev] Device fingerprinting
In-Reply-To: <58D14DAF-65DE-430C-9ACB-10E285694870@redhat.com>
References: <58D14DAF-65DE-430C-9ACB-10E285694870@redhat.com>
Message-ID: <CAJgngAdS4Q5H7OBPa+ybUjCNuT6Kqxy8RFGs1EzCSx+bPPoYUg@mail.gmail.com>

>From what I briefly read about fingerprinting and its legality it mentions
it's not that OK if used to track users, but that's not the case in this
situation I'd say, but hey I'm not a lawyer.

I wonder if we really do need anything beyond what the user agent string
gives us. Perhaps OS/Browser is sufficient?

On Tue, 18 Sep 2018 at 22:03, Douglas Palmer <dpalmer at redhat.com> wrote:

> Hi everyone
>
> I haven?t managed to find an open source solution to device fingerprinting
> which gives us everything we need. This library however gets us most of the
> way there http://valve.github.io/fingerprintjs2 <
> http://valve.github.io/fingerprintjs2>. It doesn?t give us enough
> information to distinguish between a desktop and a laptop but it will let
> us correlate devices and we can distinguish between a PC, a tablet and a
> phone. We can also get the OS, Browser and Versions from the user agent
> string.
>
> I have taken a look at a few sites the track device sessions. Apple can
> tell the difference between an iMac, a MacBook, an iPad and an iPhone.
> Facebook, GitHub, Google, LinkedIn and Pinterest don?t distinguish between
> an iMac and a MacBook. So maybe the library above is enough.
>
> I also came across the following article from the EFF which casts doubt on
> the legality of digital fingerprinting in Europe.
> https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-?
> <
> https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers
> >
>
> Does anyone have any input on any of this? Is there a better library that
> I have missed? Should we stick to parsing the user agent to avoid potential
> problems with GDPR?
>
> Regards
> Doug
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

From dpalmer at redhat.com  Wed Sep 19 13:36:57 2018
From: dpalmer at redhat.com (Douglas Palmer)
Date: Wed, 19 Sep 2018 10:36:57 -0700
Subject: [keycloak-dev] Device fingerprinting
In-Reply-To: <CAJgngAdS4Q5H7OBPa+ybUjCNuT6Kqxy8RFGs1EzCSx+bPPoYUg@mail.gmail.com>
References: <58D14DAF-65DE-430C-9ACB-10E285694870@redhat.com>
	<CAJgngAdS4Q5H7OBPa+ybUjCNuT6Kqxy8RFGs1EzCSx+bPPoYUg@mail.gmail.com>
Message-ID: <35A9633D-1FBC-4BDA-988B-68A9AF54793B@redhat.com>

The user agent will give us some device info too, it will allow us to distinguish between PC, tablet and phone in most cases.

Regards
Doug


> On Sep 19, 2018, at 10:18 AM, Stian Thorgersen <sthorger at redhat.com> wrote:
> 
> From what I briefly read about fingerprinting and its legality it mentions it's not that OK if used to track users, but that's not the case in this situation I'd say, but hey I'm not a lawyer.
> 
> I wonder if we really do need anything beyond what the user agent string gives us. Perhaps OS/Browser is sufficient?
> 
> On Tue, 18 Sep 2018 at 22:03, Douglas Palmer <dpalmer at redhat.com <mailto:dpalmer at redhat.com>> wrote:
> Hi everyone
> 
> I haven?t managed to find an open source solution to device fingerprinting which gives us everything we need. This library however gets us most of the way there http://valve.github.io/fingerprintjs2 <http://valve.github.io/fingerprintjs2> <http://valve.github.io/fingerprintjs2 <http://valve.github.io/fingerprintjs2>>. It doesn?t give us enough information to distinguish between a desktop and a laptop but it will let us correlate devices and we can distinguish between a PC, a tablet and a phone. We can also get the OS, Browser and Versions from the user agent string.
> 
> I have taken a look at a few sites the track device sessions. Apple can tell the difference between an iMac, a MacBook, an iPad and an iPhone. Facebook, GitHub, Google, LinkedIn and Pinterest don?t distinguish between an iMac and a MacBook. So maybe the library above is enough.
> 
> I also came across the following article from the EFF which casts doubt on the legality of digital fingerprinting in Europe. https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest- <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest->? <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers>>
> 
> Does anyone have any input on any of this? Is there a better library that I have missed? Should we stick to parsing the user agent to avoid potential problems with GDPR?
> 
> Regards
> Doug
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev <https://lists.jboss.org/mailman/listinfo/keycloak-dev>

From ssilvert at redhat.com  Wed Sep 19 14:11:50 2018
From: ssilvert at redhat.com (Stan Silvert)
Date: Wed, 19 Sep 2018 14:11:50 -0400
Subject: [keycloak-dev] Device fingerprinting
In-Reply-To: <35A9633D-1FBC-4BDA-988B-68A9AF54793B@redhat.com>
References: <58D14DAF-65DE-430C-9ACB-10E285694870@redhat.com>
	<CAJgngAdS4Q5H7OBPa+ybUjCNuT6Kqxy8RFGs1EzCSx+bPPoYUg@mail.gmail.com>
	<35A9633D-1FBC-4BDA-988B-68A9AF54793B@redhat.com>
Message-ID: <521e2a76-96f1-5c91-ec0e-c601d57d5484@redhat.com>

On 9/19/2018 1:36 PM, Douglas Palmer wrote:
> The user agent will give us some device info too, it will allow us to distinguish between PC, tablet and phone in most cases.
My vote is to just do everything we can with the user agent right now.? 
We parse it and make it easy to consume from the REST API. Then later, 
we figure out how to enhance it.
>
> Regards
> Doug
>
>
>> On Sep 19, 2018, at 10:18 AM, Stian Thorgersen <sthorger at redhat.com> wrote:
>>
>>  From what I briefly read about fingerprinting and its legality it mentions it's not that OK if used to track users, but that's not the case in this situation I'd say, but hey I'm not a lawyer.
>>
>> I wonder if we really do need anything beyond what the user agent string gives us. Perhaps OS/Browser is sufficient?
>>
>> On Tue, 18 Sep 2018 at 22:03, Douglas Palmer <dpalmer at redhat.com <mailto:dpalmer at redhat.com>> wrote:
>> Hi everyone
>>
>> I haven?t managed to find an open source solution to device fingerprinting which gives us everything we need. This library however gets us most of the way there http://valve.github.io/fingerprintjs2 <http://valve.github.io/fingerprintjs2> <http://valve.github.io/fingerprintjs2 <http://valve.github.io/fingerprintjs2>>. It doesn?t give us enough information to distinguish between a desktop and a laptop but it will let us correlate devices and we can distinguish between a PC, a tablet and a phone. We can also get the OS, Browser and Versions from the user agent string.
>>
>> I have taken a look at a few sites the track device sessions. Apple can tell the difference between an iMac, a MacBook, an iPad and an iPhone. Facebook, GitHub, Google, LinkedIn and Pinterest don?t distinguish between an iMac and a MacBook. So maybe the library above is enough.
>>
>> I also came across the following article from the EFF which casts doubt on the legality of digital fingerprinting in Europe. https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest- <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest->? <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers>>
>>
>> Does anyone have any input on any of this? Is there a better library that I have missed? Should we stick to parsing the user agent to avoid potential problems with GDPR?
>>
>> Regards
>> Doug
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev <https://lists.jboss.org/mailman/listinfo/keycloak-dev>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev



From sai-soma-kala.kalidindi at microfocus.com  Wed Sep 19 14:44:00 2018
From: sai-soma-kala.kalidindi at microfocus.com (Kalidindi, Sai Soma Kala)
Date: Wed, 19 Sep 2018 18:44:00 +0000
Subject: [keycloak-dev] Keycloak Error : "User with user name XXXX already
 exists. How do you want to continue"
Message-ID: <DF4PR8401MB0553B58B2009D6F70D491932EB1C0@DF4PR8401MB0553.NAMPRD84.PROD.OUTLOOK.COM>

Hi,

We are using 1.9.8 version of keycloak. We have few customers,  who has integrated their identity providers like ADFS, OKTA, Ldap ...with our Key cloak and it works. Lately we are seeing this below error message after their successful initial login, this happens to couple of users once in two weeks or so . When they try to login they see the error "User with user name XXXX already exists. How do you want to continue". Initially it used to happens once in few months, at that time,  I would go to user_entity table in keycloak and delete the entry with this username and then when user try to log in, it works and entry gets re-created . This is our fix for now, every time any user hits above error, I delete the entry form user_entity tabel.

Lately it has been happening once in few days, and we are trying to find permanent fix for this. I would like to understand why this is happening after a series of successful logins by the same user .  We are using old version of keycloak, does upgrading to latest version solve this issue? Any recommendation or help on above error is greatly appreciated.

Thanks,
Sai.

From dpalmer at redhat.com  Wed Sep 19 15:01:54 2018
From: dpalmer at redhat.com (Douglas Palmer)
Date: Wed, 19 Sep 2018 12:01:54 -0700
Subject: [keycloak-dev] Device fingerprinting
In-Reply-To: <521e2a76-96f1-5c91-ec0e-c601d57d5484@redhat.com>
References: <58D14DAF-65DE-430C-9ACB-10E285694870@redhat.com>
	<CAJgngAdS4Q5H7OBPa+ybUjCNuT6Kqxy8RFGs1EzCSx+bPPoYUg@mail.gmail.com>
	<35A9633D-1FBC-4BDA-988B-68A9AF54793B@redhat.com>
	<521e2a76-96f1-5c91-ec0e-c601d57d5484@redhat.com>
Message-ID: <C17DF3D6-D478-480C-B7B6-C548354B3FEF@redhat.com>

I think this is my preference too. If we go this route should we use ua-parser (https://github.com/ua-parser <https://github.com/ua-parser>) and maybe enhance it or should we write our own parser? Ua-parser as it stands lists my Mac as ?Other? for the device.

Regards
Doug


> On Sep 19, 2018, at 11:11 AM, Stan Silvert <ssilvert at redhat.com> wrote:
> 
> On 9/19/2018 1:36 PM, Douglas Palmer wrote:
>> The user agent will give us some device info too, it will allow us to distinguish between PC, tablet and phone in most cases.
> My vote is to just do everything we can with the user agent right now.  
> We parse it and make it easy to consume from the REST API. Then later, 
> we figure out how to enhance it.
>> 
>> Regards
>> Doug
>> 
>> 
>>> On Sep 19, 2018, at 10:18 AM, Stian Thorgersen <sthorger at redhat.com> wrote:
>>> 
>>> From what I briefly read about fingerprinting and its legality it mentions it's not that OK if used to track users, but that's not the case in this situation I'd say, but hey I'm not a lawyer.
>>> 
>>> I wonder if we really do need anything beyond what the user agent string gives us. Perhaps OS/Browser is sufficient?
>>> 
>>> On Tue, 18 Sep 2018 at 22:03, Douglas Palmer <dpalmer at redhat.com <mailto:dpalmer at redhat.com>> wrote:
>>> Hi everyone
>>> 
>>> I haven?t managed to find an open source solution to device fingerprinting which gives us everything we need. This library however gets us most of the way there http://valve.github.io/fingerprintjs2 <http://valve.github.io/fingerprintjs2> <http://valve.github.io/fingerprintjs2 <http://valve.github.io/fingerprintjs2>>. It doesn?t give us enough information to distinguish between a desktop and a laptop but it will let us correlate devices and we can distinguish between a PC, a tablet and a phone. We can also get the OS, Browser and Versions from the user agent string.
>>> 
>>> I have taken a look at a few sites the track device sessions. Apple can tell the difference between an iMac, a MacBook, an iPad and an iPhone. Facebook, GitHub, Google, LinkedIn and Pinterest don?t distinguish between an iMac and a MacBook. So maybe the library above is enough.
>>> 
>>> I also came across the following article from the EFF which casts doubt on the legality of digital fingerprinting in Europe. https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest- <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest->? <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers <https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers>>
>>> 
>>> Does anyone have any input on any of this? Is there a better library that I have missed? Should we stick to parsing the user agent to avoid potential problems with GDPR?
>>> 
>>> Regards
>>> Doug
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev <https://lists.jboss.org/mailman/listinfo/keycloak-dev>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> 
> 
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev


From sthorger at redhat.com  Thu Sep 20 02:35:45 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Thu, 20 Sep 2018 08:35:45 +0200
Subject: [keycloak-dev] Device fingerprinting
In-Reply-To: <C17DF3D6-D478-480C-B7B6-C548354B3FEF@redhat.com>
References: <58D14DAF-65DE-430C-9ACB-10E285694870@redhat.com>
	<CAJgngAdS4Q5H7OBPa+ybUjCNuT6Kqxy8RFGs1EzCSx+bPPoYUg@mail.gmail.com>
	<35A9633D-1FBC-4BDA-988B-68A9AF54793B@redhat.com>
	<521e2a76-96f1-5c91-ec0e-c601d57d5484@redhat.com>
	<C17DF3D6-D478-480C-B7B6-C548354B3FEF@redhat.com>
Message-ID: <CAJgngAf=psG8pKBo1fBSR2i2RtOF=F9_YKgd+F_v62f89TLH8g@mail.gmail.com>

+1 To just using user agent for now.

I wouldn't write our own parser, that would be a lot of effort. If we want
to do any enhancing I would do that by submitting a PR to ua-parser. I
don't think that's a priority right now. Let's get something that works,
then we can consider improving in the future based on community feedback.
Or perhaps we can even get the community to improve for us ;)

On Wed, 19 Sep 2018 at 21:02, Douglas Palmer <dpalmer at redhat.com> wrote:

> I think this is my preference too. If we go this route should we use
> ua-parser (https://github.com/ua-parser <https://github.com/ua-parser>)
> and maybe enhance it or should we write our own parser? Ua-parser as it
> stands lists my Mac as ?Other? for the device.
>
> Regards
> Doug
>
>
> > On Sep 19, 2018, at 11:11 AM, Stan Silvert <ssilvert at redhat.com> wrote:
> >
> > On 9/19/2018 1:36 PM, Douglas Palmer wrote:
> >> The user agent will give us some device info too, it will allow us to
> distinguish between PC, tablet and phone in most cases.
> > My vote is to just do everything we can with the user agent right now.
> > We parse it and make it easy to consume from the REST API. Then later,
> > we figure out how to enhance it.
> >>
> >> Regards
> >> Doug
> >>
> >>
> >>> On Sep 19, 2018, at 10:18 AM, Stian Thorgersen <sthorger at redhat.com>
> wrote:
> >>>
> >>> From what I briefly read about fingerprinting and its legality it
> mentions it's not that OK if used to track users, but that's not the case
> in this situation I'd say, but hey I'm not a lawyer.
> >>>
> >>> I wonder if we really do need anything beyond what the user agent
> string gives us. Perhaps OS/Browser is sufficient?
> >>>
> >>> On Tue, 18 Sep 2018 at 22:03, Douglas Palmer <dpalmer at redhat.com
> <mailto:dpalmer at redhat.com>> wrote:
> >>> Hi everyone
> >>>
> >>> I haven?t managed to find an open source solution to device
> fingerprinting which gives us everything we need. This library however gets
> us most of the way there http://valve.github.io/fingerprintjs2 <
> http://valve.github.io/fingerprintjs2> <
> http://valve.github.io/fingerprintjs2 <
> http://valve.github.io/fingerprintjs2>>. It doesn?t give us enough
> information to distinguish between a desktop and a laptop but it will let
> us correlate devices and we can distinguish between a PC, a tablet and a
> phone. We can also get the OS, Browser and Versions from the user agent
> string.
> >>>
> >>> I have taken a look at a few sites the track device sessions. Apple
> can tell the difference between an iMac, a MacBook, an iPad and an iPhone.
> Facebook, GitHub, Google, LinkedIn and Pinterest don?t distinguish between
> an iMac and a MacBook. So maybe the library above is enough.
> >>>
> >>> I also came across the following article from the EFF which casts
> doubt on the legality of digital fingerprinting in Europe.
> https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-
> <
> https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest->?
> <
> https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers
> <
> https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers
> >>
> >>>
> >>> Does anyone have any input on any of this? Is there a better library
> that I have missed? Should we stick to parsing the user agent to avoid
> potential problems with GDPR?
> >>>
> >>> Regards
> >>> Doug
> >>> _______________________________________________
> >>> keycloak-dev mailing list
> >>> keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev <
> https://lists.jboss.org/mailman/listinfo/keycloak-dev>
> >> _______________________________________________
> >> keycloak-dev mailing list
> >> keycloak-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >
> >
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

From sthorger at redhat.com  Thu Sep 20 10:01:02 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Thu, 20 Sep 2018 16:01:02 +0200
Subject: [keycloak-dev] Danish translation review needed
Message-ID: <CAJgngAfRj+jmhUrYz-5KyKBMjLcxeJj-rTJWc3Oj93-W01hH9w@mail.gmail.com>

Any Danish natives out there that could review
https://github.com/keycloak/keycloak/pull/5567?

From mposolda at redhat.com  Thu Sep 20 11:53:20 2018
From: mposolda at redhat.com (Marek Posolda)
Date: Thu, 20 Sep 2018 17:53:20 +0200
Subject: [keycloak-dev] Keycloak Error : "User with user name XXXX
 already exists. How do you want to continue"
In-Reply-To: <DF4PR8401MB0553B58B2009D6F70D491932EB1C0@DF4PR8401MB0553.NAMPRD84.PROD.OUTLOOK.COM>
References: <DF4PR8401MB0553B58B2009D6F70D491932EB1C0@DF4PR8401MB0553.NAMPRD84.PROD.OUTLOOK.COM>
Message-ID: <CAE8YqZ_dLTOj4AW=WXZEuJWLGJXhmWnDNjwC-5UK3uzovVoVUw@mail.gmail.com>

Hi, this version is very old and no longer supported. Upgrade to the later
version (ideally latest) is highly recommended. Marek

Dne st 19. 9. 2018 20:51 u?ivatel Kalidindi, Sai Soma Kala <
sai-soma-kala.kalidindi at microfocus.com> napsal:

> Hi,
>
> We are using 1.9.8 version of keycloak. We have few customers,  who has
> integrated their identity providers like ADFS, OKTA, Ldap ...with our Key
> cloak and it works. Lately we are seeing this below error message after
> their successful initial login, this happens to couple of users once in two
> weeks or so . When they try to login they see the error "User with user
> name XXXX already exists. How do you want to continue". Initially it used
> to happens once in few months, at that time,  I would go to user_entity
> table in keycloak and delete the entry with this username and then when
> user try to log in, it works and entry gets re-created . This is our fix
> for now, every time any user hits above error, I delete the entry form
> user_entity tabel.
>
> Lately it has been happening once in few days, and we are trying to find
> permanent fix for this. I would like to understand why this is happening
> after a series of successful logins by the same user .  We are using old
> version of keycloak, does upgrading to latest version solve this issue? Any
> recommendation or help on above error is greatly appreciated.
>
> Thanks,
> Sai.
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From thomas.darimont at googlemail.com  Mon Sep 24 05:25:46 2018
From: thomas.darimont at googlemail.com (Thomas Darimont)
Date: Mon, 24 Sep 2018 11:25:46 +0200
Subject: [keycloak-dev] Support for disabling account management for
	brokered identities
Message-ID: <CAK-7U1i=20qfOOzZKwhYiPaHoisSi7VLQ0JMxf5v_8O_DB-wZg@mail.gmail.com>

Hello Keycloak Develops,

users that are created via Identity Brokering seem to have the
account:manage-account role by default, due to the configured
default roles.

Since those accounts are usually managed by the external IdP it could
make sense to disable access to the account app for those users.

A simple way to do this is to remove the manage-account role for the account
app from those users. It would be great if the IdP configuration would
support toggling account management access (on, off).

A more generic way to do this would be to have support
for disabling the usage of default roles for user created by the IdP
whilst allowing explicit role configuration.

Do you see any problems with this?

Cheers,
Thomas

From thomas.darimont at googlemail.com  Mon Sep 24 05:37:38 2018
From: thomas.darimont at googlemail.com (Thomas Darimont)
Date: Mon, 24 Sep 2018 11:37:38 +0200
Subject: [keycloak-dev] Support for password-only sync in user federation
Message-ID: <CAK-7U1i1ngChpjEV-1+gVWKT75zK3aBDCZSbAjOpo9_PZtXn=g@mail.gmail.com>

Hello Keycloak Developers,

at the end of the recent DevNation Live session [1] A Deep Dive into
Keycloak
a user asked whether it would be possible to only sync password changes
back
with a federated user store like LDAP or Kerberos.

This would be very useful in integration scenarios where the user directory
admins
want to keep control over user profiles.

I looked at the code and it seems that one needed to add a new
UserStorageProvider.EditMode like PASSWORD_ONLY
and update the updateCredential [2] Methods accordingly to allow credential
updates.

Would this be sufficient or am I missing something?

Cheers,
Thomas

[1]
https://www.youtube.com/watch?list=PLuWlr4oKSRUZj3ax5zG_t9KE6uwTb_0rU&time_continue=1&v=ZxpY_zZ52kU
[2] org.keycloak.storage.ldap.LDAPStorageProvider#updateCredential (and
similar methods for other providers)

From sthorger at redhat.com  Mon Sep 24 14:23:44 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 24 Sep 2018 20:23:44 +0200
Subject: [keycloak-dev] Keycloak Live Events?
Message-ID: <CAJgngAf6v9CkLAoSCdRAE5NxOWNkwURKBYdTw2y=ZV1RHkmZew@mail.gmail.com>

All,

I have been considering setting up a series of live events for Keycloak.
The plan would be once a month to have a live event with presentations from
the Keycloak team and we would also be happy to invite others that want to
talk about Keycloak.

Topics would include presentations on new features, archicture/design on
upcoming features and perhaps open Q&A sessions.

Now the question is how many would attend? Let me know on the mailing list
or on Doodle (https://doodle.com/poll/qadckvmkgi6eyukd) if you are
interested. I'm also interested in knowing if you are not interested.

Suggestions for other topics are also welcome.

From sthorger at redhat.com  Mon Sep 24 14:30:54 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 24 Sep 2018 20:30:54 +0200
Subject: [keycloak-dev] Support for disabling account management for
 brokered identities
In-Reply-To: <CAK-7U1i=20qfOOzZKwhYiPaHoisSi7VLQ0JMxf5v_8O_DB-wZg@mail.gmail.com>
References: <CAK-7U1i=20qfOOzZKwhYiPaHoisSi7VLQ0JMxf5v_8O_DB-wZg@mail.gmail.com>
Message-ID: <CAJgngAcaMj_L8GHP7xmoPu9qxLDm8ZNT4wwz+JqhrZs6jwoTjQ@mail.gmail.com>

A switch to include default roles that is enabled by default would be
better. That way you can choose to add all default roles or to not add them
and manage roles in the IdP mappers instead.

On Mon, 24 Sep 2018 at 11:31, Thomas Darimont <
thomas.darimont at googlemail.com> wrote:

> Hello Keycloak Develops,
>
> users that are created via Identity Brokering seem to have the
> account:manage-account role by default, due to the configured
> default roles.
>
> Since those accounts are usually managed by the external IdP it could
> make sense to disable access to the account app for those users.
>
> A simple way to do this is to remove the manage-account role for the
> account
> app from those users. It would be great if the IdP configuration would
> support toggling account management access (on, off).
>
> A more generic way to do this would be to have support
> for disabling the usage of default roles for user created by the IdP
> whilst allowing explicit role configuration.
>
> Do you see any problems with this?
>
> Cheers,
> Thomas
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From sthorger at redhat.com  Mon Sep 24 14:32:56 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Mon, 24 Sep 2018 20:32:56 +0200
Subject: [keycloak-dev] Support for password-only sync in user federation
In-Reply-To: <CAK-7U1i1ngChpjEV-1+gVWKT75zK3aBDCZSbAjOpo9_PZtXn=g@mail.gmail.com>
References: <CAK-7U1i1ngChpjEV-1+gVWKT75zK3aBDCZSbAjOpo9_PZtXn=g@mail.gmail.com>
Message-ID: <CAJgngAfO_6P3FTmZnrkFgB6tVP-hiX6bgLQQEaPtYNRj8d_HUA@mail.gmail.com>

I thought the question was to allow password changes with read-only and my
assumption was that he wanted the change password in Keycloak only.

I'm no expert on the LDAP integration, but I believe you can control what
attributes are written back to LDAP in the protocol mappers. So could you
not achieve what you're thinking with simply setting all mappers to
read-only?

On Mon, 24 Sep 2018 at 11:43, Thomas Darimont <
thomas.darimont at googlemail.com> wrote:

> Hello Keycloak Developers,
>
> at the end of the recent DevNation Live session [1] A Deep Dive into
> Keycloak
> a user asked whether it would be possible to only sync password changes
> back
> with a federated user store like LDAP or Kerberos.
>
> This would be very useful in integration scenarios where the user directory
> admins
> want to keep control over user profiles.
>
> I looked at the code and it seems that one needed to add a new
> UserStorageProvider.EditMode like PASSWORD_ONLY
> and update the updateCredential [2] Methods accordingly to allow credential
> updates.
>
> Would this be sufficient or am I missing something?
>
> Cheers,
> Thomas
>
> [1]
>
> https://www.youtube.com/watch?list=PLuWlr4oKSRUZj3ax5zG_t9KE6uwTb_0rU&time_continue=1&v=ZxpY_zZ52kU
> [2] org.keycloak.storage.ldap.LDAPStorageProvider#updateCredential (and
> similar methods for other providers)
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>

From mposolda at redhat.com  Tue Sep 25 02:45:34 2018
From: mposolda at redhat.com (Marek Posolda)
Date: Tue, 25 Sep 2018 08:45:34 +0200
Subject: [keycloak-dev] Support for password-only sync in user federation
In-Reply-To: <CAJgngAfO_6P3FTmZnrkFgB6tVP-hiX6bgLQQEaPtYNRj8d_HUA@mail.gmail.com>
References: <CAK-7U1i1ngChpjEV-1+gVWKT75zK3aBDCZSbAjOpo9_PZtXn=g@mail.gmail.com>
	<CAJgngAfO_6P3FTmZnrkFgB6tVP-hiX6bgLQQEaPtYNRj8d_HUA@mail.gmail.com>
Message-ID: <7710e45e-de09-cada-00d9-4f6177f18123@redhat.com>

Yes, exactly.

For LDAP, you can already achieve this. You just need to make sure that 
LDAP provider is configured with WRITABLE edit mode and then mappers for 
various attributes (firstName, lastName, email) are configured with 
"Read Only" switch ON and "Always Read from LDAP" switch to OFF. That 
way, if you update user profile in Keycloak, the updates will go just to 
the Keycloak DB, not to LDAP. And Keycloak will read the values from DB 
with bigger preference than from LDAP. However when password is written 
in Keycloak, it will be updated in LDAP and also password verifications 
will be triggered against LDAP. I've just tried it and works as expected.

For Kerberos Provider, we don't yet have support for updating password. 
This will require implementation of "Kerberos Password Update" protocol. 
We have JIRA already opened for it (We had PR for this some time ago, 
but it added bunch of ApacheDS dependencies, so we couldn't accept it).

For custom UserStorage providers written by you, you don't need separate 
editMode as well. In this case, you have control over your 
implementation and you can implement updates and reads exactly how you want.

IMO there is no need to introduce another EditMode value.

Marek


On 24/09/18 20:32, Stian Thorgersen wrote:
> I thought the question was to allow password changes with read-only and my
> assumption was that he wanted the change password in Keycloak only.
>
> I'm no expert on the LDAP integration, but I believe you can control what
> attributes are written back to LDAP in the protocol mappers. So could you
> not achieve what you're thinking with simply setting all mappers to
> read-only?
>
> On Mon, 24 Sep 2018 at 11:43, Thomas Darimont <
> thomas.darimont at googlemail.com> wrote:
>
>> Hello Keycloak Developers,
>>
>> at the end of the recent DevNation Live session [1] A Deep Dive into
>> Keycloak
>> a user asked whether it would be possible to only sync password changes
>> back
>> with a federated user store like LDAP or Kerberos.
>>
>> This would be very useful in integration scenarios where the user directory
>> admins
>> want to keep control over user profiles.
>>
>> I looked at the code and it seems that one needed to add a new
>> UserStorageProvider.EditMode like PASSWORD_ONLY
>> and update the updateCredential [2] Methods accordingly to allow credential
>> updates.
>>
>> Would this be sufficient or am I missing something?
>>
>> Cheers,
>> Thomas
>>
>> [1]
>>
>> https://www.youtube.com/watch?list=PLuWlr4oKSRUZj3ax5zG_t9KE6uwTb_0rU&time_continue=1&v=ZxpY_zZ52kU
>> [2] org.keycloak.storage.ldap.LDAPStorageProvider#updateCredential (and
>> similar methods for other providers)
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev



From mposolda at redhat.com  Tue Sep 25 02:49:44 2018
From: mposolda at redhat.com (Marek Posolda)
Date: Tue, 25 Sep 2018 08:49:44 +0200
Subject: [keycloak-dev] Support for disabling account management for
 brokered identities
In-Reply-To: <CAJgngAcaMj_L8GHP7xmoPu9qxLDm8ZNT4wwz+JqhrZs6jwoTjQ@mail.gmail.com>
References: <CAK-7U1i=20qfOOzZKwhYiPaHoisSi7VLQ0JMxf5v_8O_DB-wZg@mail.gmail.com>
	<CAJgngAcaMj_L8GHP7xmoPu9qxLDm8ZNT4wwz+JqhrZs6jwoTjQ@mail.gmail.com>
Message-ID: <fe531666-88cd-ada1-dfee-6f6a6643dea4@redhat.com>

+1

IMO this switch could be added to "CreateUserIfUnique" authenticator 
used for the FirstBroker flow. That's the one, which is responsible for 
creating new users.

Marek

On 24/09/18 20:30, Stian Thorgersen wrote:
> A switch to include default roles that is enabled by default would be
> better. That way you can choose to add all default roles or to not add them
> and manage roles in the IdP mappers instead.
>
> On Mon, 24 Sep 2018 at 11:31, Thomas Darimont <
> thomas.darimont at googlemail.com> wrote:
>
>> Hello Keycloak Develops,
>>
>> users that are created via Identity Brokering seem to have the
>> account:manage-account role by default, due to the configured
>> default roles.
>>
>> Since those accounts are usually managed by the external IdP it could
>> make sense to disable access to the account app for those users.
>>
>> A simple way to do this is to remove the manage-account role for the
>> account
>> app from those users. It would be great if the IdP configuration would
>> support toggling account management access (on, off).
>>
>> A more generic way to do this would be to have support
>> for disabling the usage of default roles for user created by the IdP
>> whilst allowing explicit role configuration.
>>
>> Do you see any problems with this?
>>
>> Cheers,
>> Thomas
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev



From sthorger at redhat.com  Tue Sep 25 02:52:30 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Tue, 25 Sep 2018 08:52:30 +0200
Subject: [keycloak-dev] Support for disabling account management for
 brokered identities
In-Reply-To: <fe531666-88cd-ada1-dfee-6f6a6643dea4@redhat.com>
References: <CAK-7U1i=20qfOOzZKwhYiPaHoisSi7VLQ0JMxf5v_8O_DB-wZg@mail.gmail.com>
	<CAJgngAcaMj_L8GHP7xmoPu9qxLDm8ZNT4wwz+JqhrZs6jwoTjQ@mail.gmail.com>
	<fe531666-88cd-ada1-dfee-6f6a6643dea4@redhat.com>
Message-ID: <CAJgngAdnW7vuzoEPaazd4bBmD9hChxwzMgu3KYJkk7Un+3zvaA@mail.gmail.com>

I think it should rather be an option on the IdP itself as you may want to
have different settings for different IdPs. Adding it to the first broker
flow would be for all or nothing.

On Tue, 25 Sep 2018 at 08:49, Marek Posolda <mposolda at redhat.com> wrote:

> +1
>
> IMO this switch could be added to "CreateUserIfUnique" authenticator
> used for the FirstBroker flow. That's the one, which is responsible for
> creating new users.
>
> Marek
>
> On 24/09/18 20:30, Stian Thorgersen wrote:
> > A switch to include default roles that is enabled by default would be
> > better. That way you can choose to add all default roles or to not add
> them
> > and manage roles in the IdP mappers instead.
> >
> > On Mon, 24 Sep 2018 at 11:31, Thomas Darimont <
> > thomas.darimont at googlemail.com> wrote:
> >
> >> Hello Keycloak Develops,
> >>
> >> users that are created via Identity Brokering seem to have the
> >> account:manage-account role by default, due to the configured
> >> default roles.
> >>
> >> Since those accounts are usually managed by the external IdP it could
> >> make sense to disable access to the account app for those users.
> >>
> >> A simple way to do this is to remove the manage-account role for the
> >> account
> >> app from those users. It would be great if the IdP configuration would
> >> support toggling account management access (on, off).
> >>
> >> A more generic way to do this would be to have support
> >> for disabling the usage of default roles for user created by the IdP
> >> whilst allowing explicit role configuration.
> >>
> >> Do you see any problems with this?
> >>
> >> Cheers,
> >> Thomas
> >> _______________________________________________
> >> keycloak-dev mailing list
> >> keycloak-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>
> > _______________________________________________
> > keycloak-dev mailing list
> > keycloak-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>
>

From mposolda at redhat.com  Tue Sep 25 03:01:28 2018
From: mposolda at redhat.com (Marek Posolda)
Date: Tue, 25 Sep 2018 09:01:28 +0200
Subject: [keycloak-dev] Support for disabling account management for
 brokered identities
In-Reply-To: <CAJgngAdnW7vuzoEPaazd4bBmD9hChxwzMgu3KYJkk7Un+3zvaA@mail.gmail.com>
References: <CAK-7U1i=20qfOOzZKwhYiPaHoisSi7VLQ0JMxf5v_8O_DB-wZg@mail.gmail.com>
	<CAJgngAcaMj_L8GHP7xmoPu9qxLDm8ZNT4wwz+JqhrZs6jwoTjQ@mail.gmail.com>
	<fe531666-88cd-ada1-dfee-6f6a6643dea4@redhat.com>
	<CAJgngAdnW7vuzoEPaazd4bBmD9hChxwzMgu3KYJkk7Un+3zvaA@mail.gmail.com>
Message-ID: <8dad709a-cfeb-3bec-d333-df24cf9f6fb3@redhat.com>

I don't have very strong preference, feel free to add to Idp if you 
thing it's better. But I am slightly more keen to have it rather on the 
authenticator. Logically it belongs here IMO as that makes sure to 
create new users.

If you really want to have different settings for different IdPs, you 
can create different "First Broker Login" flows for different brokers 
and configure them different way. Similarly like you can do today if you 
want "Facebook" users to immediately update their password after they're 
created, but "Other IDP" users to not require update password (As the 
switch for "Update Password After Import" is also defined on the 
Authenticator).

Marek

On 25/09/18 08:52, Stian Thorgersen wrote:
> I think it should rather be an option on the IdP itself as you may 
> want to have different settings for different IdPs. Adding it to the 
> first broker flow would be for all or nothing.
>
> On Tue, 25 Sep 2018 at 08:49, Marek Posolda <mposolda at redhat.com 
> <mailto:mposolda at redhat.com>> wrote:
>
>     +1
>
>     IMO this switch could be added to "CreateUserIfUnique" authenticator
>     used for the FirstBroker flow. That's the one, which is
>     responsible for
>     creating new users.
>
>     Marek
>
>     On 24/09/18 20:30, Stian Thorgersen wrote:
>     > A switch to include default roles that is enabled by default
>     would be
>     > better. That way you can choose to add all default roles or to
>     not add them
>     > and manage roles in the IdP mappers instead.
>     >
>     > On Mon, 24 Sep 2018 at 11:31, Thomas Darimont <
>     > thomas.darimont at googlemail.com
>     <mailto:thomas.darimont at googlemail.com>> wrote:
>     >
>     >> Hello Keycloak Develops,
>     >>
>     >> users that are created via Identity Brokering seem to have the
>     >> account:manage-account role by default, due to the configured
>     >> default roles.
>     >>
>     >> Since those accounts are usually managed by the external IdP it
>     could
>     >> make sense to disable access to the account app for those users.
>     >>
>     >> A simple way to do this is to remove the manage-account role
>     for the
>     >> account
>     >> app from those users. It would be great if the IdP
>     configuration would
>     >> support toggling account management access (on, off).
>     >>
>     >> A more generic way to do this would be to have support
>     >> for disabling the usage of default roles for user created by
>     the IdP
>     >> whilst allowing explicit role configuration.
>     >>
>     >> Do you see any problems with this?
>     >>
>     >> Cheers,
>     >> Thomas
>     >> _______________________________________________
>     >> keycloak-dev mailing list
>     >> keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
>     >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>     >>
>     > _______________________________________________
>     > keycloak-dev mailing list
>     > keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
>     > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>


From sthorger at redhat.com  Wed Sep 26 05:17:24 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Wed, 26 Sep 2018 11:17:24 +0200
Subject: [keycloak-dev] Support for disabling account management for
 brokered identities
In-Reply-To: <8dad709a-cfeb-3bec-d333-df24cf9f6fb3@redhat.com>
References: <CAK-7U1i=20qfOOzZKwhYiPaHoisSi7VLQ0JMxf5v_8O_DB-wZg@mail.gmail.com>
	<CAJgngAcaMj_L8GHP7xmoPu9qxLDm8ZNT4wwz+JqhrZs6jwoTjQ@mail.gmail.com>
	<fe531666-88cd-ada1-dfee-6f6a6643dea4@redhat.com>
	<CAJgngAdnW7vuzoEPaazd4bBmD9hChxwzMgu3KYJkk7Un+3zvaA@mail.gmail.com>
	<8dad709a-cfeb-3bec-d333-df24cf9f6fb3@redhat.com>
Message-ID: <CAJgngAdAApoiyddYxjD2LxJp1xxz9qOHEJbFEiaTj_SQMgiq0w@mail.gmail.com>

DIdn't think about having different flows for different IdPs. That's a
better option, so +1 to making it an option in the flow rather than IdPs
directly.

On Tue, 25 Sep 2018 at 09:01, Marek Posolda <mposolda at redhat.com> wrote:

> I don't have very strong preference, feel free to add to Idp if you thing
> it's better. But I am slightly more keen to have it rather on the
> authenticator. Logically it belongs here IMO as that makes sure to create
> new users.
>
> If you really want to have different settings for different IdPs, you can
> create different "First Broker Login" flows for different brokers and
> configure them different way. Similarly like you can do today if you want
> "Facebook" users to immediately update their password after they're
> created, but "Other IDP" users to not require update password (As the
> switch for "Update Password After Import" is also defined on the
> Authenticator).
>
> Marek
>
> On 25/09/18 08:52, Stian Thorgersen wrote:
>
> I think it should rather be an option on the IdP itself as you may want to
> have different settings for different IdPs. Adding it to the first broker
> flow would be for all or nothing.
>
> On Tue, 25 Sep 2018 at 08:49, Marek Posolda <mposolda at redhat.com> wrote:
>
>> +1
>>
>> IMO this switch could be added to "CreateUserIfUnique" authenticator
>> used for the FirstBroker flow. That's the one, which is responsible for
>> creating new users.
>>
>> Marek
>>
>> On 24/09/18 20:30, Stian Thorgersen wrote:
>> > A switch to include default roles that is enabled by default would be
>> > better. That way you can choose to add all default roles or to not add
>> them
>> > and manage roles in the IdP mappers instead.
>> >
>> > On Mon, 24 Sep 2018 at 11:31, Thomas Darimont <
>> > thomas.darimont at googlemail.com> wrote:
>> >
>> >> Hello Keycloak Develops,
>> >>
>> >> users that are created via Identity Brokering seem to have the
>> >> account:manage-account role by default, due to the configured
>> >> default roles.
>> >>
>> >> Since those accounts are usually managed by the external IdP it could
>> >> make sense to disable access to the account app for those users.
>> >>
>> >> A simple way to do this is to remove the manage-account role for the
>> >> account
>> >> app from those users. It would be great if the IdP configuration would
>> >> support toggling account management access (on, off).
>> >>
>> >> A more generic way to do this would be to have support
>> >> for disabling the usage of default roles for user created by the IdP
>> >> whilst allowing explicit role configuration.
>> >>
>> >> Do you see any problems with this?
>> >>
>> >> Cheers,
>> >> Thomas
>> >> _______________________________________________
>> >> keycloak-dev mailing list
>> >> keycloak-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> >>
>> > _______________________________________________
>> > keycloak-dev mailing list
>> > keycloak-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>>
>>
>

From sthorger at redhat.com  Wed Sep 26 15:12:15 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Wed, 26 Sep 2018 21:12:15 +0200
Subject: [keycloak-dev] Feature freeze for 4.x
Message-ID: <CAJgngAeqXVm64VZtZ3-+fiWnjMSU=d3nnh2_ftOo1E4tKH1GrA@mail.gmail.com>

We are nearing the completion of 4.x and are entering into a feature
freeze.

We will try to get through the current backlog of PRs, please be proactive
and answer any feedback we give on GitHub.

Anyone that wants to contribute additional features and enhancements to 4.x
should do so very soon, otherwise we are most likely not able to accept
until we start on 5.x.

>From November and most likely until end of January the team will focus on
bug fixing, automation and improvements to our testsuite. In this period
I'm afraid we are not able to accept new features or enhancements, but
please do send contributions regardless. We will review and add it to the
queue for things to be merged once we can open up the gates again.

I'm hoping that in the future with the effort we put in now on automation
and testsuite improvements we will not have to have such lengthy yearly
features freezes. Next time around we should be talking about weeks not
months.

As a final note thanks to everyone that has contributed to Keycloak. Be it
in the form of code, documentation or simply answering questions on the
mailing list. The community is what it is all about and we are very prod to
have such a great community around Keycloak.

From sthorger at redhat.com  Thu Sep 27 02:59:34 2018
From: sthorger at redhat.com (Stian Thorgersen)
Date: Thu, 27 Sep 2018 08:59:34 +0200
Subject: [keycloak-dev] Keycloak 4.5.0.Final is out
Message-ID: <CAJgngAes3+jX1gsRuvKDv4uBk52N6b-T9N8WRi+rjjxJbfYUQA@mail.gmail.com>

http://blog.keycloak.org/2018/09/keycloak-450final-released.html

From Volker.Suschke at bosch-si.com  Thu Sep 27 10:08:18 2018
From: Volker.Suschke at bosch-si.com (Suschke Volker (INST-CSS/BSV-OS))
Date: Thu, 27 Sep 2018 14:08:18 +0000
Subject: [keycloak-dev] Pagination and search on groups at group membership
	view
Message-ID: <76cabc10431541ce83020d3ef1501848@bosch-si.com>

Hi Keycloak team,

as already discussed in https://issues.jboss.org/browse/KEYCLOAK-2538 we also have a large number of groups which need to be managed in group membership UI. Current situation is not satisfactory since two lists with possible thousands of groups need to be displayed. Pull request https://github.com/keycloak/keycloak/pull/4710 is already opened for a long time and handles pagination and search for available groups part of this view only.
Our plan is to extend REST API to support pagination and search at /groups endpoint on user resource and, as well both parts of group membership UI. What do you think about our plans?

Mit freundlichen Gr??en / Best regards

Volker Suschke

(INST-CSS/BSV-OS)
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-157 | Mobil +49 174 2763164 | Fax +49 30 726112-100 | volker.suschke at bosch-si.com<mailto:volker.suschke at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn



From sergey at shimkiv.com  Thu Sep 27 14:42:51 2018
From: sergey at shimkiv.com (Serhii Shymkiv)
Date: Thu, 27 Sep 2018 21:42:51 +0300
Subject: [keycloak-dev] [keycloak-user] Keycloak 4.5.0.Final is out
In-Reply-To: <CAJgngAes3+jX1gsRuvKDv4uBk52N6b-T9N8WRi+rjjxJbfYUQA@mail.gmail.com>
References: <CAJgngAes3+jX1gsRuvKDv4uBk52N6b-T9N8WRi+rjjxJbfYUQA@mail.gmail.com>
Message-ID: <CAD7UjAPpemu4=9Twa9K2T612K5+A8DogT9rwND90sGMimk=9Cw@mail.gmail.com>

As always, thank you !
And the quick one regarding the updated versions of the Maven artifacts -
they are on the way, right ?



On Thu, Sep 27, 2018 at 10:02 AM Stian Thorgersen <sthorger at redhat.com>
wrote:

> http://blog.keycloak.org/2018/09/keycloak-450final-released.html
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user
>


-- 
Best regards,
Serhii Shymkiv.

From sergey at shimkiv.com  Fri Sep 28 03:12:12 2018
From: sergey at shimkiv.com (Serhii Shymkiv)
Date: Fri, 28 Sep 2018 10:12:12 +0300
Subject: [keycloak-dev] [keycloak-user] Keycloak 4.5.0.Final is out
In-Reply-To: <CAD7UjAPpemu4=9Twa9K2T612K5+A8DogT9rwND90sGMimk=9Cw@mail.gmail.com>
References: <CAJgngAes3+jX1gsRuvKDv4uBk52N6b-T9N8WRi+rjjxJbfYUQA@mail.gmail.com>
	<CAD7UjAPpemu4=9Twa9K2T612K5+A8DogT9rwND90sGMimk=9Cw@mail.gmail.com>
Message-ID: <CAD7UjAOFXdCc2qovRHWMTqwQ1HhYJh5=Y5vbEEmcdhV55fmiCw@mail.gmail.com>

No worries, artifacts already updated in central repository.

On Thu, Sep 27, 2018 at 9:42 PM Serhii Shymkiv <sergey at shimkiv.com> wrote:

> As always, thank you !
> And the quick one regarding the updated versions of the Maven artifacts -
> they are on the way, right ?
>
>
>
> On Thu, Sep 27, 2018 at 10:02 AM Stian Thorgersen <sthorger at redhat.com>
> wrote:
>
>> http://blog.keycloak.org/2018/09/keycloak-450final-released.html
>> _______________________________________________
>> keycloak-user mailing list
>> keycloak-user at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-user
>>
>
>
> --
> Best regards,
> Serhii Shymkiv.
>


-- 
Best regards,
Serhii Shymkiv.

From alistair.doswald at elca.ch  Fri Sep 28 08:33:05 2018
From: alistair.doswald at elca.ch (Doswald Alistair)
Date: Fri, 28 Sep 2018 12:33:05 +0000
Subject: [keycloak-dev] Implementation of artifact binding (JIRA
	KEYCLOAK-831)
Message-ID: <dc527510d6c042e68fdbb6ecd1f14167@elca.ch>

Implementation of artifact binding (JIRA KEYCLOAK-831)

Hello,

Last week I did a PoC implementation of the SAML artifact binding in a branch off keycloak 4.3.0.Final. The implementation can be seen here at https://github.com/AlistairDoswald/keycloak/tree/projectathon (don't judge me too harshly for the quality of the code if you look at it, I had about 2 days to have a working implementation, which included finding out how that part of the protocol worked).

However, I now want to write a "correct" implementation against keycloak/master and if possible I'd like some feedback/advice on my intended implementation.


1. General implementation

>From the description in the SAML specification (see here section 3.6, https://docs.oasis-open.org/security/saml/v2.0/saml-bindings-2.0-os.pdf), artifact binding can be used for transmitting the request message, the response message or both.

Initially, I intend only to do the implementation for the response messages. If I'm not mistaken, this means only for the Response and LogoutResponse messages. Would this be considered a suitable implementation of the JIRA?


2. User interface

When a SP requests an artifact, it can do so by specifying HTTP-Artifact instead of HTTP-POST or HTTP-redirect, and the process is then transparent with regard to the configuration of the client. However, I believe that the client should have a "Force artifact binding" binary slider and also a field to specify an artifact binding address. In this manner, the artifact binding can be used in conjunction with the IdP initiated login method.

Importing must also set the artifact binding address if it is present in the SP metadata.


3. IdP metadata

IdP metadata must contain at least one ArtifactResolutionService, I intend to have only one, with its index set to 0 and isDefault=true, and the binding set to the same address as the HTTP-POST (as for ECP)


4. Sending an artifact instead of the normal saml message

This is the section for which I have the greatest uncertainty with respect to a correct implementation.

Broadly this means intercepting the output response, and sending a 302 redirect or a POSTed form with the artifact instead. Considering the length of the artifact, I see no reason to use a form, but should this be an option in the GUI?

More practically, this means generating the response, saving it in the cache, and sending the redirect (or form) instead. I believe that the client's cache would be the best place to save this information (through the AuthenticatedClientSessionModel to be precise), but I'm not certain because it's the first time I'm seeking to store some new information in the cache. The key would be the artifact, and the value in my view should be the document, as that way we can create a complete signed/encrypted ArtifactResponse containing the Response or LogoutResponse.

For the implementation details I'm not sure if it would be best to make the changes directly in the SamlProtocol class, or to do something similar to the SamlECPProfileService which overrides the methods of the SamlProtocol. For SamlECPProfileService the current implementation makes sense, but for artifact binding I fear there would be significant code duplication (of course, I could also do a mix with some small modifications in the SamlProtocol class and a SamlArtifactProfileService, or something similar).

For triggering this artifact workflow, it would either be if the AuthnRequest has a ProtocolBinding set to urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Artifact, or if the client has "force artifact binding" set to true.


5. Receiving an ArtifactResolve message

For this part, my current implementation seems correct to me: the soapBinding method in class SamlService is modified to check the contents of the soap message arriving: if it is an ArtifactResolve, the corresponding ArtifactResponse generated earlier is packaged in a soap message and sent as a response. If not, the ECP profile is tried.

The key-ArtifactResponse pair is removed from the cache during this operation. I am, however, not sure yet how the cache should handle purging of expired ArtifactResponse messages that are never asked for.


6. Errors, logging and audit

Obviously, the error handling should work as described in the protocol, but also be logged as such. I don't think there's any messages to log in INFO, but the DEBUG logs should show the messages and allow an admin to easily put the entire sequence together.

Also, I don't think there's any need for any extra information in the audit logs.


7. Tests

Obviously, I'll have to add some tests for these functions, which should be:

- Standard unit tests for individual functions that can be separated from objects that would otherwise have to be mocked
- Tests with arquillian to test the flow with artifact binding (sp initiated and idp initiated), the options available in the GUI (extra field, forced) as well as the error cases (i.e. asking twice for the same artifact, for an artifact that doesn't exist, etc...).


If you have any comments (anything missing, things that should be implemented differently in your view, etc...) feel free to let me know.

Best regards,

Alistair Doswald



From Sebastian.Schuster at bosch-si.com  Fri Sep 28 10:00:03 2018
From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST-CSS/BSV-OS))
Date: Fri, 28 Sep 2018 14:00:03 +0000
Subject: [keycloak-dev] Latest changes with JGroups discovery in 4.5.0.Final
	Docker Image
Message-ID: <24738023a02b48f2b974d5f211fdf96c@bosch-si.com>

Hi everybody,

I think there are some minor issues with the changes in the 4.5.0 Docker image. In docker-entrypoint.sh per default if nothing is specified the jboss.bind.address and jboss.bind.address.private are both set to hostname ?i and if nothing is specified standalone-ha mode is used. I find that at least questionable, I think running standalone is a safer default compared to opening JGroups communication on a public interface. However, the default works for us in Kubernetes.

However, the detection whether a profile was specified (if echo "$@" | egrep -v -- "-c "; then)  should be improved,  only looking for ?-c? does not work as ??server-config? is equally possible. Wildfly will die with an error if both are present?

Best regards,
Sebastian


Mit freundlichen Gr??en / Best regards

Dr.-Ing. Sebastian Schuster

Open Source Services (INST-CSS/BSV-OS)
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn




From Sebastian.Schuster at bosch-si.com  Fri Sep 28 11:42:12 2018
From: Sebastian.Schuster at bosch-si.com (Schuster Sebastian (INST-CSS/BSV-OS))
Date: Fri, 28 Sep 2018 15:42:12 +0000
Subject: [keycloak-dev] Latest changes with JGroups discovery in
 4.5.0.Final	Docker Image
In-Reply-To: <24738023a02b48f2b974d5f211fdf96c@bosch-si.com>
References: <24738023a02b48f2b974d5f211fdf96c@bosch-si.com>
Message-ID: <eaf8c8d352454958985a382a03755b1b@bosch-si.com>

Maybe this snippet is helpful:

if echo "$@" | egrep -v -- '-c |-c=|--server-config |--server-config='; then
    SYS_PROPS+=" -c=standalone-ha.xml"
fi

Best regards,
Sebastian

Mit freundlichen Gr??en / Best regards

Dr.-Ing.  Sebastian Schuster

Open Source Services (INST-CSS/BSV-OS) 
Bosch?Software Innovations?GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B 
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn 




-----Original Message-----
From: keycloak-dev-bounces at lists.jboss.org <keycloak-dev-bounces at lists.jboss.org> On Behalf Of Schuster Sebastian (INST-CSS/BSV-OS)
Sent: Freitag, 28. September 2018 16:00
To: keycloak-dev <keycloak-dev at lists.jboss.org>
Subject: [keycloak-dev] Latest changes with JGroups discovery in 4.5.0.Final Docker Image

Hi everybody,

I think there are some minor issues with the changes in the 4.5.0 Docker image. In docker-entrypoint.sh per default if nothing is specified the jboss.bind.address and jboss.bind.address.private are both set to hostname ?i and if nothing is specified standalone-ha mode is used. I find that at least questionable, I think running standalone is a safer default compared to opening JGroups communication on a public interface. However, the default works for us in Kubernetes.

However, the detection whether a profile was specified (if echo "$@" | egrep -v -- "-c "; then)  should be improved,  only looking for ?-c? does not work as ??server-config? is equally possible. Wildfly will die with an error if both are present?

Best regards,
Sebastian


Mit freundlichen Gr??en / Best regards

Dr.-Ing. Sebastian Schuster

Open Source Services (INST-CSS/BSV-OS)
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-485 | Fax +49 30 726112-100 | Sebastian.Schuster at bosch-si.com<mailto:Sebastian.Schuster at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten L?cke; Gesch?ftsf?hrung: Dr. Stefan Ferber, Michael Hahn



_______________________________________________
keycloak-dev mailing list
keycloak-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev