From mposolda at redhat.com Tue Nov 3 02:44:09 2015 From: mposolda at redhat.com (Marek Posolda) Date: Tue, 3 Nov 2015 08:44:09 +0100 Subject: [keycloak-dev] redesign of federation In-Reply-To: <5633EB83.3000806@redhat.com> References: <5633EB83.3000806@redhat.com> Message-ID: <563865C9.6030407@redhat.com> On 30/10/15 23:13, Bill Burke wrote: > In doing group model, I was thinking more about federation. Our SPI > kinda sucks. I was thinking that local storage (Model API) and > UserFederation should be the same exact SPI. Instead of just > RealmProvider and UserProvider, we might break it up into: > > * RealmProvider - holds realms and clients > * UserProvider - holds username and attributes about the user > * UserRelationshipProvider - holds user role mappings, user group membership > * UserCredentialProvider - stores and authenticates credentials > * GroupProvider - holds group definitions > * RoleProvider - holds role definitions +1 for adding GroupProvider and RoleProvider. That seems to me like big limitation of current federation provider, that it's not able to handle CRUD of roles into the external storage. But I might be missing why we need 3 separate "user" providers? It seems that current model can handle well the user attributes, relationships and credentials? Once UserFederationProvider returns the proxy object, you can implement CRUD of relationships and credentials in the proxy object itself? > > One of the big problems we have is that roles and groups have to be > defined within Keycloak DB even though they might live in one or more > external stores. > > Admin console would have to change too. You'd have to pick which > database you wanted to manage. i.e. if you wanted to add a role you > might want to add it to an LDAP store and not local storage. Hmm... shouldn't we have same pattern like for user federation? So the role and group will be always added to "local" DB, but additionally synced to the external DB? Thing is that external storage may not be able to save all the metadata we have for roles and groups. Also we need some reference (roleID, groupID) to be available in Keycloak. From my LDAP experience, the CRUD of relationships is not a problem, as you can return UserModel proxy where you can override the relationship CRUD methods (grantRole, deleteRoleMapping, getRoleMappings, hasRole...). The problem is the CRUD of roles itself. When you create new role in Keycloak admin console, it's saved into LDAP during first call of "grantRole" . So this direction is quite fine. But the LDAP to Keycloak direction is much worse. In other words,when new role is added in LDAP, there is no way to notify Keycloak about it. Marek > > This is something we'd really have to map out and design. I would love > to be able to do it before product, but I don't think we'll have enough > time to bake it in community. Maybe something we'll have to wait for > Keycloak 2.0. > From sthorger at redhat.com Tue Nov 3 03:31:43 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 3 Nov 2015 08:31:43 +0000 Subject: [keycloak-dev] Plan for "First login with identity brokers" In-Reply-To: <56338541.5060403@redhat.com> References: <562F36E8.9060501@redhat.com> <562F7693.80805@redhat.com> <562F9286.5020205@redhat.com> <563130DB.2090604@redhat.com> <5631E9FA.1050302@redhat.com> <5632211D.5020706@redhat.com> <56338541.5060403@redhat.com> Message-ID: Would be even simpler for users if we just removed authentication completely and only had the username on the login form - we could just add a statement "only use your own username, we trust you to not try to login as someone else" ;) Seriously though - social accounts are hacked all the time and allowing this auto linking of accounts without requiring users to authenticate to the existing account is just plain scary. The solution to the use case you've given is not login with another social provider, it's having good account recovery options in place. On 30 October 2015 at 14:57, Bill Burke wrote: > There's an alternative problem. Logs in with Twitter in 2005. Logs in > again 2015 with Google. Is required to link with Twitter, says "screw it" > because he doesn't remember his Twitter password and just closes his > browser and doesn't use the website. > > I've been on really popular high-traffic sites where their google login > was broken for months (mmqb.si.com which is an NFL website for Sports > Illustrated). I used my Facebook identity instead. If I had been required > to merge accounts manually, I would have not been able to use the site. > > On 10/29/2015 4:35 PM, Stian Thorgersen wrote: > >> Linking accounts automatically is fine, but we should not have an option >> that can do that without requiring users to authenticate first. >> >> There are so many cases where a user could have one social account >> compromised. They may not care that much about the account, they may >> never use the service so they've completely forgotten about it. >> >> Imagine the following scenario: >> >> * Tom signed up for GMail in 2005 - figured it was great and continued >> using the service the rest of his life >> * Tom signed up for Twitter in 2005 - figured it was not to his taste >> and never used the account again >> * Tom now read about two factor auth and configured it on his GMail >> account >> * Mary (a bad person) figured that the password to Toms twitter account >> was 'password' so she's gained access to Tom's Twitter - Tom doesn't >> know, but he doesn't care either >> * Tom signs up for a website that uses Keycloak and logs in with his >> trusted GMail account >> * Now if we let Mary login to the website that uses Keycloak with Toms >> old Twitter account, without first proving she's Tom (which she can't), >> would be just plain daft! >> >> On 29 October 2015 at 06:37, Bill Burke > > wrote: >> >> >> >> On 10/29/2015 5:42 AM, Vlastimil Elias wrote: >> > >> > >> > On 28.10.2015 21:32, Bill Burke wrote: >> >> If a user has loads of social networks and links a bunch of them, >> if >> >> *any one* of them is compromised the entire account is compromised. >> >> Most sites using social login, the only reason is there is a login >> is >> >> for the appliation to collect marketing data. So, the default >> behavior >> >> should make things as simple as possible for the user. >> >> >> >> At a minimum, by default, the user should not be required to link >> an >> >> account if there is a conflicting duplicate email given by the >> provider. >> >> I have founddeveloeprs.redhat.com >> very difficult >> to use. >> > >> > yep, it is difficult to use because it have to follow company's >> policy >> > with unique emails and Keycloak do not provide necessary support for >> > simple and user friendly account linking currently ;-) >> > >> >> Yeah, its not your fault. Its ours. >> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151103/62fb7839/attachment-0001.html From mposolda at redhat.com Tue Nov 3 06:24:44 2015 From: mposolda at redhat.com (Marek Posolda) Date: Tue, 3 Nov 2015 12:24:44 +0100 Subject: [keycloak-dev] Plan for "First login with identity brokers" In-Reply-To: References: <562F36E8.9060501@redhat.com> <562F7693.80805@redhat.com> <562F9286.5020205@redhat.com> <563130DB.2090604@redhat.com> <5631E9FA.1050302@redhat.com> <5632211D.5020706@redhat.com> <56338541.5060403@redhat.com> Message-ID: <5638997C.9040600@redhat.com> I have a prototype in progress, which I am going to present on Thursday call. It's based on authentication SPI, so it's quite flexible . Current default behaviour is, when it detects duplicated email, it displays the page with "Duplication detected. What do you want to do?" Then user can: - Go back and edit the profile. So user is not required to link provider as long as he provides different unique email - Link the provider. At this point, he need either to reauthenticate by different way (password+otp or already linked identity provider) or confirm the linking via email Marek On 03/11/15 09:31, Stian Thorgersen wrote: > Would be even simpler for users if we just removed authentication > completely and only had the username on the login form - we could just > add a statement "only use your own username, we trust you to not try > to login as someone else" ;) > > Seriously though - social accounts are hacked all the time and > allowing this auto linking of accounts without requiring users to > authenticate to the existing account is just plain scary. > > The solution to the use case you've given is not login with another > social provider, it's having good account recovery options in place. > > On 30 October 2015 at 14:57, Bill Burke > wrote: > > There's an alternative problem. Logs in with Twitter in 2005. > Logs in again 2015 with Google. Is required to link with Twitter, > says "screw it" because he doesn't remember his Twitter password > and just closes his browser and doesn't use the website. > > I've been on really popular high-traffic sites where their google > login was broken for months (mmqb.si.com > which is an NFL website for Sports Illustrated). I used my > Facebook identity instead. If I had been required to merge > accounts manually, I would have not been able to use the site. > > On 10/29/2015 4:35 PM, Stian Thorgersen wrote: > > Linking accounts automatically is fine, but we should not have > an option > that can do that without requiring users to authenticate first. > > There are so many cases where a user could have one social account > compromised. They may not care that much about the account, > they may > never use the service so they've completely forgotten about it. > > Imagine the following scenario: > > * Tom signed up for GMail in 2005 - figured it was great and > continued > using the service the rest of his life > * Tom signed up for Twitter in 2005 - figured it was not to > his taste > and never used the account again > * Tom now read about two factor auth and configured it on his > GMail account > * Mary (a bad person) figured that the password to Toms > twitter account > was 'password' so she's gained access to Tom's Twitter - Tom > doesn't > know, but he doesn't care either > * Tom signs up for a website that uses Keycloak and logs in > with his > trusted GMail account > * Now if we let Mary login to the website that uses Keycloak > with Toms > old Twitter account, without first proving she's Tom (which > she can't), > would be just plain daft! > > On 29 October 2015 at 06:37, Bill Burke > >> wrote: > > > > On 10/29/2015 5:42 AM, Vlastimil Elias wrote: > > > > > > On 28.10.2015 21:32, Bill Burke wrote: > >> If a user has loads of social networks and links a > bunch of them, if > >> *any one* of them is compromised the entire account is > compromised. > >> Most sites using social login, the only reason is there > is a login is > >> for the appliation to collect marketing data. So, the > default behavior > >> should make things as simple as possible for the user. > >> > >> At a minimum, by default, the user should not be > required to link an > >> account if there is a conflicting duplicate email given > by the provider. > >> I have founddeveloeprs.redhat.com > > very difficult > to use. > > > > yep, it is difficult to use because it have to follow > company's policy > > with unique emails and Keycloak do not provide necessary > support for > > simple and user friendly account linking currently ;-) > > > > Yeah, its not your fault. Its ours. > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151103/d224bb24/attachment.html From sthorger at redhat.com Tue Nov 3 07:18:06 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 3 Nov 2015 13:18:06 +0100 Subject: [keycloak-dev] Plan for "First login with identity brokers" In-Reply-To: <5638997C.9040600@redhat.com> References: <562F36E8.9060501@redhat.com> <562F7693.80805@redhat.com> <562F9286.5020205@redhat.com> <563130DB.2090604@redhat.com> <5631E9FA.1050302@redhat.com> <5632211D.5020706@redhat.com> <56338541.5060403@redhat.com> <5638997C.9040600@redhat.com> Message-ID: Sounds good On 3 November 2015 at 12:24, Marek Posolda wrote: > I have a prototype in progress, which I am going to present on Thursday > call. It's based on authentication SPI, so it's quite flexible . > > Current default behaviour is, when it detects duplicated email, it > displays the page with "Duplication detected. What do you want to do?" Then > user can: > - Go back and edit the profile. So user is not required to link provider > as long as he provides different unique email > - Link the provider. At this point, he need either to reauthenticate by > different way (password+otp or already linked identity provider) or confirm > the linking via email > > Marek > > > On 03/11/15 09:31, Stian Thorgersen wrote: > > Would be even simpler for users if we just removed authentication > completely and only had the username on the login form - we could just add > a statement "only use your own username, we trust you to not try to login > as someone else" ;) > > Seriously though - social accounts are hacked all the time and allowing > this auto linking of accounts without requiring users to authenticate to > the existing account is just plain scary. > > The solution to the use case you've given is not login with another social > provider, it's having good account recovery options in place. > > On 30 October 2015 at 14:57, Bill Burke wrote: > >> There's an alternative problem. Logs in with Twitter in 2005. Logs in >> again 2015 with Google. Is required to link with Twitter, says "screw it" >> because he doesn't remember his Twitter password and just closes his >> browser and doesn't use the website. >> >> I've been on really popular high-traffic sites where their google login >> was broken for months (mmqb.si.com which is an NFL website for Sports >> Illustrated). I used my Facebook identity instead. If I had been required >> to merge accounts manually, I would have not been able to use the site. >> >> On 10/29/2015 4:35 PM, Stian Thorgersen wrote: >> >>> Linking accounts automatically is fine, but we should not have an option >>> that can do that without requiring users to authenticate first. >>> >>> There are so many cases where a user could have one social account >>> compromised. They may not care that much about the account, they may >>> never use the service so they've completely forgotten about it. >>> >>> Imagine the following scenario: >>> >>> * Tom signed up for GMail in 2005 - figured it was great and continued >>> using the service the rest of his life >>> * Tom signed up for Twitter in 2005 - figured it was not to his taste >>> and never used the account again >>> * Tom now read about two factor auth and configured it on his GMail >>> account >>> * Mary (a bad person) figured that the password to Toms twitter account >>> was 'password' so she's gained access to Tom's Twitter - Tom doesn't >>> know, but he doesn't care either >>> * Tom signs up for a website that uses Keycloak and logs in with his >>> trusted GMail account >>> * Now if we let Mary login to the website that uses Keycloak with Toms >>> old Twitter account, without first proving she's Tom (which she can't), >>> would be just plain daft! >>> >>> On 29 October 2015 at 06:37, Bill Burke < >>> bburke at redhat.com >>> > wrote: >>> >>> >>> >>> On 10/29/2015 5:42 AM, Vlastimil Elias wrote: >>> > >>> > >>> > On 28.10.2015 21:32, Bill Burke wrote: >>> >> If a user has loads of social networks and links a bunch of them, >>> if >>> >> *any one* of them is compromised the entire account is >>> compromised. >>> >> Most sites using social login, the only reason is there is a >>> login is >>> >> for the appliation to collect marketing data. So, the default >>> behavior >>> >> should make things as simple as possible for the user. >>> >> >>> >> At a minimum, by default, the user should not be required to link >>> an >>> >> account if there is a conflicting duplicate email given by the >>> provider. >>> >> I have founddeveloeprs.redhat.com < >>> http://develoeprs.redhat.com> very difficult >>> to use. >>> > >>> > yep, it is difficult to use because it have to follow company's >>> policy >>> > with unique emails and Keycloak do not provide necessary support >>> for >>> > simple and user friendly account linking currently ;-) >>> > >>> >>> Yeah, its not your fault. Its ours. >>> >>> >>> -- >>> Bill Burke >>> JBoss, a division of Red Hat >>> http://bill.burkecentral.com >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> > > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151103/ab09d5ed/attachment-0001.html From bburke at redhat.com Tue Nov 3 09:19:55 2015 From: bburke at redhat.com (Bill Burke) Date: Tue, 3 Nov 2015 09:19:55 -0500 Subject: [keycloak-dev] redesign of federation In-Reply-To: <563865C9.6030407@redhat.com> References: <5633EB83.3000806@redhat.com> <563865C9.6030407@redhat.com> Message-ID: <5638C28B.5090600@redhat.com> On 11/3/2015 2:44 AM, Marek Posolda wrote: > On 30/10/15 23:13, Bill Burke wrote: >> In doing group model, I was thinking more about federation. Our SPI >> kinda sucks. I was thinking that local storage (Model API) and >> UserFederation should be the same exact SPI. Instead of just >> RealmProvider and UserProvider, we might break it up into: >> >> * RealmProvider - holds realms and clients >> * UserProvider - holds username and attributes about the user >> * UserRelationshipProvider - holds user role mappings, user group >> membership >> * UserCredentialProvider - stores and authenticates credentials >> * GroupProvider - holds group definitions >> * RoleProvider - holds role definitions > +1 for adding GroupProvider and RoleProvider. That seems to me like big > limitation of current federation provider, that it's not able to handle > CRUD of roles into the external storage. > > But I might be missing why we need 3 separate "user" providers? It seems > that current model can handle well the user attributes, relationships > and credentials? Once UserFederationProvider returns the proxy object, > you can implement CRUD of relationships and credentials in the proxy > object itself? >> >> One of the big problems we have is that roles and groups have to be >> defined within Keycloak DB even though they might live in one or more >> external stores. >> >> Admin console would have to change too. You'd have to pick which >> database you wanted to manage. i.e. if you wanted to add a role you >> might want to add it to an LDAP store and not local storage. > Hmm... shouldn't we have same pattern like for user federation? So the > role and group will be always added to "local" DB, but additionally > synced to the external DB? Thing is that external storage may not be > able to save all the metadata we have for roles and groups. Also we need > some reference (roleID, groupID) to be available in Keycloak. > I want to avoid syncing. Hence, pulling roles/groups out of realm DB. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Tue Nov 3 09:36:04 2015 From: bburke at redhat.com (Bill Burke) Date: Tue, 3 Nov 2015 09:36:04 -0500 Subject: [keycloak-dev] Plan for "First login with identity brokers" In-Reply-To: References: <562F36E8.9060501@redhat.com> <562F7693.80805@redhat.com> <562F9286.5020205@redhat.com> <563130DB.2090604@redhat.com> <5631E9FA.1050302@redhat.com> <5632211D.5020706@redhat.com> <56338541.5060403@redhat.com> Message-ID: <5638C654.6060703@redhat.com> On 11/3/2015 3:31 AM, Stian Thorgersen wrote: > Would be even simpler for users if we just removed authentication > completely and only had the username on the login form - we could just > add a statement "only use your own username, we trust you to not try to > login as someone else" ;) > > Seriously though - social accounts are hacked all the time and allowing > this auto linking of accounts without requiring users to authenticate to > the existing account is just plain scary. > Sensitive sites do not use social login to protect their users. > The solution to the use case you've given is not login with another > social provider, it's having good account recovery options in place. > I hope the argument here is just what the default should be. IMO, the default should be that a new account per social provider is created and email duplicates are allowed. User would manually merge accounts via the account service panel if they want. These features are equally important IMO: * Broker providers should be able to be automatically trusted with a switch and automatically merge accounts. * brokers should have a flow attached to them so that they can require account merging and such. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Tue Nov 3 09:54:47 2015 From: bburke at redhat.com (Bill Burke) Date: Tue, 3 Nov 2015 09:54:47 -0500 Subject: [keycloak-dev] Plan for "First login with identity brokers" In-Reply-To: References: <562F36E8.9060501@redhat.com> <562F7693.80805@redhat.com> <562F9286.5020205@redhat.com> <563130DB.2090604@redhat.com> <5631E9FA.1050302@redhat.com> <5632211D.5020706@redhat.com> <56338541.5060403@redhat.com> <5638997C.9040600@redhat.com> Message-ID: <5638CAB7.7080309@redhat.com> -100. The default should be to create a duplicate account. On 11/3/2015 7:18 AM, Stian Thorgersen wrote: > Sounds good > > On 3 November 2015 at 12:24, Marek Posolda > wrote: > > I have a prototype in progress, which I am going to present on > Thursday call. It's based on authentication SPI, so it's quite > flexible . > > Current default behaviour is, when it detects duplicated email, it > displays the page with "Duplication detected. What do you want to > do?" Then user can: > - Go back and edit the profile. So user is not required to link > provider as long as he provides different unique email > - Link the provider. At this point, he need either to reauthenticate > by different way (password+otp or already linked identity provider) > or confirm the linking via email > > Marek > > > On 03/11/15 09:31, Stian Thorgersen wrote: >> Would be even simpler for users if we just removed authentication >> completely and only had the username on the login form - we could >> just add a statement "only use your own username, we trust you to >> not try to login as someone else" ;) >> >> Seriously though - social accounts are hacked all the time and >> allowing this auto linking of accounts without requiring users to >> authenticate to the existing account is just plain scary. >> >> The solution to the use case you've given is not login with >> another social provider, it's having good account recovery options >> in place. >> >> On 30 October 2015 at 14:57, Bill Burke > > wrote: >> >> There's an alternative problem. Logs in with Twitter in >> 2005. Logs in again 2015 with Google. Is required to link >> with Twitter, says "screw it" because he doesn't remember his >> Twitter password and just closes his browser and doesn't use >> the website. >> >> I've been on really popular high-traffic sites where their >> google login was broken for months (mmqb.si.com >> which is an NFL website for Sports >> Illustrated). I used my Facebook identity instead. If I had >> been required to merge accounts manually, I would have not >> been able to use the site. >> >> On 10/29/2015 4:35 PM, Stian Thorgersen wrote: >> >> Linking accounts automatically is fine, but we should not >> have an option >> that can do that without requiring users to authenticate >> first. >> >> There are so many cases where a user could have one social >> account >> compromised. They may not care that much about the >> account, they may >> never use the service so they've completely forgotten >> about it. >> >> Imagine the following scenario: >> >> * Tom signed up for GMail in 2005 - figured it was great >> and continued >> using the service the rest of his life >> * Tom signed up for Twitter in 2005 - figured it was not >> to his taste >> and never used the account again >> * Tom now read about two factor auth and configured it on >> his GMail account >> * Mary (a bad person) figured that the password to Toms >> twitter account >> was 'password' so she's gained access to Tom's Twitter - >> Tom doesn't >> know, but he doesn't care either >> * Tom signs up for a website that uses Keycloak and logs >> in with his >> trusted GMail account >> * Now if we let Mary login to the website that uses >> Keycloak with Toms >> old Twitter account, without first proving she's Tom >> (which she can't), >> would be just plain daft! >> >> On 29 October 2015 at 06:37, Bill Burke >> <bburke at redhat.com >> >> >> wrote: >> >> >> >> On 10/29/2015 5:42 AM, Vlastimil Elias wrote: >> > >> > >> > On 28.10.2015 21:32, Bill Burke wrote: >> >> If a user has loads of social networks and links a >> bunch of them, if >> >> *any one* of them is compromised the entire account >> is compromised. >> >> Most sites using social login, the only reason is >> there is a login is >> >> for the appliation to collect marketing data. So, >> the default behavior >> >> should make things as simple as possible for the user. >> >> >> >> At a minimum, by default, the user should not be >> required to link an >> >> account if there is a conflicting duplicate email >> given by the provider. >> >> I have founddeveloeprs.redhat.com >> >> very difficult >> to use. >> > >> > yep, it is difficult to use because it have to >> follow company's policy >> > with unique emails and Keycloak do not provide >> necessary support for >> > simple and user friendly account linking currently ;-) >> > >> >> Yeah, its not your fault. Its ours. >> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> >> > > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> >> >> >> >> _______________________________________________ keycloak-dev >> mailing list keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Tue Nov 3 16:04:43 2015 From: bburke at redhat.com (Bill Burke) Date: Tue, 3 Nov 2015 16:04:43 -0500 Subject: [keycloak-dev] roles vs. groups Message-ID: <5639216B.9050500@redhat.com> Stian and I were having a conversation about roles, keycloak composite roles, vs. groups. It seems that groups and roles are often confused and one can be equivalent to the other. One common thread is the following: Groups are user centric. Roles are permission centric. "A group is a means of organising users, whereas a role is usually a means of organising rights." So, keycloak composite roles would be a way to organise rights for a set of applications. For example, you might have a set of sales services, each sales service has an "admin" and "user" role. You'd define a "sales user" and "sales admin" role which would be a composite containing the "admin" and/or "user" role of each sales service. Conversely, a keycloak group would provide a way to organize a set of users. You would create a group called "sales associates" add members to it and then assign the roles members of the group can partake. Really, in Keycloak with composite roles, you can have a role act as a group. So, while groups and roles are logically the sameAdding the concept of a group though provides distinction and clarity without overloading the concept of a composite. So, given that, Role mapping tab for Groups and Users would be named "Permissions" instead of "Role Mappings". Each role would have a "Rights" tab instead of the "Composite Role" concept we have now. That might bring more clarity? Or will it just confuse concepts that are going to be introduced by Pedro and his Authz stuff? I'm also thinking that a Groups and Role Namespaces could be combined. So a group would have a set of "Permissions" (role mappings) that are automatically granted to user members. The group could also define a set of "Roles" that apply to this group. So "Sales" could have a "Manager" role. This "Manager" role would be a composite role that assigns additional permissions. This would also allow us to define default roles for -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Tue Nov 3 16:08:12 2015 From: bburke at redhat.com (Bill Burke) Date: Tue, 3 Nov 2015 16:08:12 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <5639216B.9050500@redhat.com> References: <5639216B.9050500@redhat.com> Message-ID: <5639223C.3020403@redhat.com> On 11/3/2015 4:04 PM, Bill Burke wrote: > Stian and I were having a conversation about roles, keycloak composite > roles, vs. groups. It seems that groups and roles are often confused > and one can be equivalent to the other. One common thread is the following: > > Groups are user centric. Roles are permission centric. > > "A group is a means of organising users, whereas a role is usually a > means of organising rights." > > So, keycloak composite roles would be a way to organise rights for a set > of applications. For example, you might have a set of sales services, > each sales service has an "admin" and "user" role. You'd define a > "sales user" and "sales admin" role which would be a composite > containing the "admin" and/or "user" role of each sales service. > > Conversely, a keycloak group would provide a way to organize a set of > users. You would create a group called "sales associates" add members > to it and then assign the roles members of the group can partake. > > Really, in Keycloak with composite roles, you can have a role act as a > group. So, while groups and roles are logically the sameAdding the > concept of a group though provides distinction and clarity without > overloading the concept of a composite. > > So, given that, Role mapping tab for Groups and Users would be named > "Permissions" instead of "Role Mappings". Each role would have a > "Rights" tab instead of the "Composite Role" concept we have now. That > might bring more clarity? Or will it just confuse concepts that are > going to be introduced by Pedro and his Authz stuff? > > I'm also thinking that a Groups and Role Namespaces could be combined. > So a group would have a set of "Permissions" (role mappings) that are > automatically granted to user members. The group could also define a > set of "Roles" that apply to this group. So "Sales" could have a > "Manager" role. This "Manager" role would be a composite role that > assigns additional permissions. This would also allow us to define > default roles for > Whoops, I didn't finish.... Combining Groups and Role Namespaces would allow us to define built in roles for the Group that when assigned would allow management of the members of the group. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Tue Nov 3 16:20:58 2015 From: bburke at redhat.com (Bill Burke) Date: Tue, 3 Nov 2015 16:20:58 -0500 Subject: [keycloak-dev] user groups vs. client groups Message-ID: <5639253A.7090408@redhat.com> In my previous email I talked about combining Groups and Role Namespaces. Now I want to talk about User Groups vs. Client Groups. User Groups would manage a set of users. Members would automatically inherit a set of "permissions": a set of roles. User Groups would also provide a set of attributes that the user inherits. I'd like to introduce the concept of a Client Group. Client Group would have: * Roles - basically a role namespace * Permissions - set of roles service accounts members inherit * Scope - same as our current concept of scope * Protocol Policies - common protocol configuration. Each Client Group would have some default roles defined. i.e. roles that allow a user to edit any client in the client group. Each Client would have the same configuration options. They would be able to have an additional set of roles, permissions, scope, and overridable Protocol Policies. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From jorsol at gmail.com Tue Nov 3 16:33:07 2015 From: jorsol at gmail.com (=?UTF-8?Q?Jorge_Sol=C3=B3rzano?=) Date: Tue, 3 Nov 2015 15:33:07 -0600 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <5639216B.9050500@redhat.com> References: <5639216B.9050500@redhat.com> Message-ID: I think the concepts should be standardized: Permissions: are the most atomic level of a security policy and they are statements of functionality. Can you open a door? Can you read a file? Can you delete a customer record? Can you push a button? Roles: are effectively a collection of permissions used to simplify the management of permissions and users. So users can be assigned roles instead of being assigned permissions directly, which can get complicated with larger user bases and more complex applications. So, for example, a bank application might have an administrator role or a bank teller role. Users: A user is the "who" of an application. Groups: Is a collection of users and define a set of roles/permisions, users are members of groups. The asociation for me is something like this: Groups can have Roles and/or Permisions asociated to it. User can have Roles and Permisions and can be members of Groups, by inheritance users that are members of groups have all the permisions asociated to the groups. Roles can have one ore more permissions, this are explicit permisions. There should be deny permisions too. Jorge Sol?rzano From psilva at redhat.com Tue Nov 3 17:01:57 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Tue, 3 Nov 2015 17:01:57 -0500 (EST) Subject: [keycloak-dev] roles vs. groups In-Reply-To: <5639216B.9050500@redhat.com> References: <5639216B.9050500@redhat.com> Message-ID: <1447226686.2954301.1446588117003.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Bill Burke" > To: keycloak-dev at lists.jboss.org > Sent: Tuesday, November 3, 2015 7:04:43 PM > Subject: [keycloak-dev] roles vs. groups > > Stian and I were having a conversation about roles, keycloak composite > roles, vs. groups. It seems that groups and roles are often confused > and one can be equivalent to the other. One common thread is the following: > > Groups are user centric. Roles are permission centric. > > "A group is a means of organising users, whereas a role is usually a > means of organising rights." IMO, groups can also be used to organize rights. It is quite common to only check if an user is member of a group or any of its parents. Despite any role ... In any case, handle groups and roles separately is the best way to go ... > > So, keycloak composite roles would be a way to organise rights for a set > of applications. For example, you might have a set of sales services, > each sales service has an "admin" and "user" role. You'd define a > "sales user" and "sales admin" role which would be a composite > containing the "admin" and/or "user" role of each sales service. > > Conversely, a keycloak group would provide a way to organize a set of > users. You would create a group called "sales associates" add members > to it and then assign the roles members of the group can partake. > > Really, in Keycloak with composite roles, you can have a role act as a > group. So, while groups and roles are logically the sameAdding the > concept of a group though provides distinction and clarity without > overloading the concept of a composite. +1 Regarding "role act as a group". Yes, you can. And that is what JEE gives you and it is not enough when you really need to support the group concept, specially when you need to consider group hierarchies, permission inherited to members, etc. You need some role mapping hack to get all that working. IMO, Role (or composite role) != Group ... > > So, given that, Role mapping tab for Groups and Users would be named > "Permissions" instead of "Role Mappings". Each role would have a > "Rights" tab instead of the "Composite Role" concept we have now. That > might bring more clarity? Or will it just confuse concepts that are > going to be introduced by Pedro and his Authz stuff? Basically, what you are doing is group and role mapping in the end. Where you can add "permissions" to an group or an user. The Authz stuff will just use whatever you define in order to write policies for your protected resources. For instance, we can now define a Group-Based Policy, so you can say that all members of a group (or any of its parents) can access a resource or perform some operation on it. Or even say that any member with a role X within a group can do something. > > I'm also thinking that a Groups and Role Namespaces could be combined. > So a group would have a set of "Permissions" (role mappings) that are > automatically granted to user members. The group could also define a > set of "Roles" that apply to this group. So "Sales" could have a > "Manager" role. This "Manager" role would be a composite role that > assigns additional permissions. This would also allow us to define > default roles for >Whoops, I didn't finish.... >Combining Groups and Role Namespaces would allow us to define built in >roles for the Group that when assigned would allow management of the >members of the group. If I got this properly, what you mean is support "GroupRole" relationships. Where you may say that Bill is Manager in Group A. IIRC, Hawkular requires that. > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From psilva at redhat.com Tue Nov 3 17:13:23 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Tue, 3 Nov 2015 17:13:23 -0500 (EST) Subject: [keycloak-dev] roles vs. groups In-Reply-To: References: <5639216B.9050500@redhat.com> Message-ID: <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Jorge Sol?rzano" > To: "Bill Burke" > Cc: keycloak-dev at lists.jboss.org > Sent: Tuesday, November 3, 2015 7:33:07 PM > Subject: Re: [keycloak-dev] roles vs. groups > > I think the concepts should be standardized: > > Permissions: are the most atomic level of a security policy and they > are statements of functionality. Can you open a door? Can you read a > file? Can you delete a customer record? Can you push a button? > > Roles: are effectively a collection of permissions used to simplify > the management of permissions and users. So users can be assigned > roles instead of being assigned permissions directly, which can get > complicated with larger user bases and more complex applications. So, > for example, a bank application might have an administrator role or a > bank teller role. > > Users: A user is the "who" of an application. > > Groups: Is a collection of users and define a set of roles/permisions, > users are members of groups. > > The asociation for me is something like this: > Groups can have Roles and/or Permisions asociated to it. > User can have Roles and Permisions and can be members of Groups, by > inheritance users that are members of groups have all the permisions > asociated to the groups. > Roles can have one ore more permissions, this are explicit permisions. > > There should be deny permisions too. Don't you think that positive logic is enough ? > > > Jorge Sol?rzano > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From jorsol at gmail.com Tue Nov 3 19:02:38 2015 From: jorsol at gmail.com (=?UTF-8?Q?Jorge_Sol=C3=B3rzano?=) Date: Tue, 3 Nov 2015 18:02:38 -0600 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> Message-ID: Let's supose, you have a group called "GroupA", that group have roles "Create invoice" this has 13 permisions, "Remove invoice" this has 5 permisions, "Update invoice" this has 19 permisions... I asign 25 users to Group A, but 2 users, should not have 3 permisions that are different in both users, should I need to create "GroupB" and "GroupC" with the exact permissions, just to handle this 3 permisions exclusions? It probably can be a little overkill, but IHMO is more flexible than an all or nothing approach. Jorge Sol?rzano On Tue, Nov 3, 2015 at 4:13 PM, Pedro Igor Silva wrote: > ----- Original Message ----- >> From: "Jorge Sol?rzano" >> To: "Bill Burke" >> Cc: keycloak-dev at lists.jboss.org >> Sent: Tuesday, November 3, 2015 7:33:07 PM >> Subject: Re: [keycloak-dev] roles vs. groups >> >> I think the concepts should be standardized: >> >> Permissions: are the most atomic level of a security policy and they >> are statements of functionality. Can you open a door? Can you read a >> file? Can you delete a customer record? Can you push a button? >> >> Roles: are effectively a collection of permissions used to simplify >> the management of permissions and users. So users can be assigned >> roles instead of being assigned permissions directly, which can get >> complicated with larger user bases and more complex applications. So, >> for example, a bank application might have an administrator role or a >> bank teller role. >> >> Users: A user is the "who" of an application. >> >> Groups: Is a collection of users and define a set of roles/permisions, >> users are members of groups. >> >> The asociation for me is something like this: >> Groups can have Roles and/or Permisions asociated to it. >> User can have Roles and Permisions and can be members of Groups, by >> inheritance users that are members of groups have all the permisions >> asociated to the groups. >> Roles can have one ore more permissions, this are explicit permisions. >> >> There should be deny permisions too. > > Don't you think that positive logic is enough ? > >> >> >> Jorge Sol?rzano >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev From bburke at redhat.com Tue Nov 3 20:13:07 2015 From: bburke at redhat.com (Bill Burke) Date: Tue, 3 Nov 2015 20:13:07 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <1447226686.2954301.1446588117003.JavaMail.zimbra@redhat.com> References: <5639216B.9050500@redhat.com> <1447226686.2954301.1446588117003.JavaMail.zimbra@redhat.com> Message-ID: <56395BA3.6020509@redhat.com> On 11/3/2015 5:01 PM, Pedro Igor Silva wrote: > ----- Original Message ----- >> From: "Bill Burke" >> To: keycloak-dev at lists.jboss.org >> Sent: Tuesday, November 3, 2015 7:04:43 PM >> Subject: [keycloak-dev] roles vs. groups >> >> Stian and I were having a conversation about roles, keycloak composite >> roles, vs. groups. It seems that groups and roles are often confused >> and one can be equivalent to the other. One common thread is the following: >> >> Groups are user centric. Roles are permission centric. >> >> "A group is a means of organising users, whereas a role is usually a >> means of organising rights." > > IMO, groups can also be used to organize rights. It is quite common to only check if an user is member of a group or any of its parents. Despite any role ... > > In any case, handle groups and roles separately is the best way to go ... > >> >> So, keycloak composite roles would be a way to organise rights for a set >> of applications. For example, you might have a set of sales services, >> each sales service has an "admin" and "user" role. You'd define a >> "sales user" and "sales admin" role which would be a composite >> containing the "admin" and/or "user" role of each sales service. >> >> Conversely, a keycloak group would provide a way to organize a set of >> users. You would create a group called "sales associates" add members >> to it and then assign the roles members of the group can partake. >> >> Really, in Keycloak with composite roles, you can have a role act as a >> group. So, while groups and roles are logically the sameAdding the >> concept of a group though provides distinction and clarity without >> overloading the concept of a composite. > > +1 > > Regarding "role act as a group". Yes, you can. And that is what JEE gives you and it is not enough when you really need to support the group concept, specially when you need to consider group hierarchies, permission inherited to members, etc. You need some role mapping hack to get all that working. > > IMO, Role (or composite role) != Group ... > >> >> So, given that, Role mapping tab for Groups and Users would be named >> "Permissions" instead of "Role Mappings". Each role would have a >> "Rights" tab instead of the "Composite Role" concept we have now. That >> might bring more clarity? Or will it just confuse concepts that are >> going to be introduced by Pedro and his Authz stuff? > > Basically, what you are doing is group and role mapping in the end. Where you can add "permissions" to an group or an user. > > The Authz stuff will just use whatever you define in order to write policies for your protected resources. For instance, we can now define a Group-Based Policy, so you can say that all members of a group (or any of its parents) can access a resource or perform some operation on it. > > Or even say that any member with a role X within a group can do something. > >> >> I'm also thinking that a Groups and Role Namespaces could be combined. >> So a group would have a set of "Permissions" (role mappings) that are >> automatically granted to user members. The group could also define a >> set of "Roles" that apply to this group. So "Sales" could have a >> "Manager" role. This "Manager" role would be a composite role that >> assigns additional permissions. This would also allow us to define >> default roles for > >> Whoops, I didn't finish.... >> Combining Groups and Role Namespaces would allow us to define built in >> roles for the Group that when assigned would allow management of the >> members of the group. > > If I got this properly, what you mean is support "GroupRole" relationships. Where you may say that Bill is Manager in Group A. > > IIRC, Hawkular requires that. > I'm just proposing a role namespace. i.e. Group A defines a role called "Manager". Bill has role mapping A.Manager. I want to avoid overlapping with what you are doing. Each group would have some built in declared roles: i.e. add-user, manage-user, manage-group, etc...So that a user can be permitted to manage the members of the group. A few people have been asking for this. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From ssilvert at redhat.com Tue Nov 3 20:24:47 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Tue, 03 Nov 2015 20:24:47 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <56395BA3.6020509@redhat.com> References: <5639216B.9050500@redhat.com> <1447226686.2954301.1446588117003.JavaMail.zimbra@redhat.com> <56395BA3.6020509@redhat.com> Message-ID: <56395E5F.7040705@redhat.com> I was just about to respond to this thread when I realized that it's kind of pointless without diagrams. Wish we could all be in Brno right now with a shared whiteboard. On 11/3/2015 8:13 PM, Bill Burke wrote: > > On 11/3/2015 5:01 PM, Pedro Igor Silva wrote: >> ----- Original Message ----- >>> From: "Bill Burke" >>> To: keycloak-dev at lists.jboss.org >>> Sent: Tuesday, November 3, 2015 7:04:43 PM >>> Subject: [keycloak-dev] roles vs. groups >>> >>> Stian and I were having a conversation about roles, keycloak composite >>> roles, vs. groups. It seems that groups and roles are often confused >>> and one can be equivalent to the other. One common thread is the following: >>> >>> Groups are user centric. Roles are permission centric. >>> >>> "A group is a means of organising users, whereas a role is usually a >>> means of organising rights." >> IMO, groups can also be used to organize rights. It is quite common to only check if an user is member of a group or any of its parents. Despite any role ... >> >> In any case, handle groups and roles separately is the best way to go ... >> >>> So, keycloak composite roles would be a way to organise rights for a set >>> of applications. For example, you might have a set of sales services, >>> each sales service has an "admin" and "user" role. You'd define a >>> "sales user" and "sales admin" role which would be a composite >>> containing the "admin" and/or "user" role of each sales service. >>> >>> Conversely, a keycloak group would provide a way to organize a set of >>> users. You would create a group called "sales associates" add members >>> to it and then assign the roles members of the group can partake. >>> >>> Really, in Keycloak with composite roles, you can have a role act as a >>> group. So, while groups and roles are logically the sameAdding the >>> concept of a group though provides distinction and clarity without >>> overloading the concept of a composite. >> +1 >> >> Regarding "role act as a group". Yes, you can. And that is what JEE gives you and it is not enough when you really need to support the group concept, specially when you need to consider group hierarchies, permission inherited to members, etc. You need some role mapping hack to get all that working. >> >> IMO, Role (or composite role) != Group ... >> >>> So, given that, Role mapping tab for Groups and Users would be named >>> "Permissions" instead of "Role Mappings". Each role would have a >>> "Rights" tab instead of the "Composite Role" concept we have now. That >>> might bring more clarity? Or will it just confuse concepts that are >>> going to be introduced by Pedro and his Authz stuff? >> Basically, what you are doing is group and role mapping in the end. Where you can add "permissions" to an group or an user. >> >> The Authz stuff will just use whatever you define in order to write policies for your protected resources. For instance, we can now define a Group-Based Policy, so you can say that all members of a group (or any of its parents) can access a resource or perform some operation on it. >> >> Or even say that any member with a role X within a group can do something. >> >>> I'm also thinking that a Groups and Role Namespaces could be combined. >>> So a group would have a set of "Permissions" (role mappings) that are >>> automatically granted to user members. The group could also define a >>> set of "Roles" that apply to this group. So "Sales" could have a >>> "Manager" role. This "Manager" role would be a composite role that >>> assigns additional permissions. This would also allow us to define >>> default roles for >>> Whoops, I didn't finish.... >>> Combining Groups and Role Namespaces would allow us to define built in >>> roles for the Group that when assigned would allow management of the >>> members of the group. >> If I got this properly, what you mean is support "GroupRole" relationships. Where you may say that Bill is Manager in Group A. >> >> IIRC, Hawkular requires that. >> > I'm just proposing a role namespace. i.e. Group A defines a role called > "Manager". Bill has role mapping A.Manager. I want to avoid > overlapping with what you are doing. Each group would have some built > in declared roles: i.e. add-user, manage-user, manage-group, etc...So > that a user can be permitted to manage the members of the group. A few > people have been asking for this. > > > From mposolda at redhat.com Wed Nov 4 02:26:46 2015 From: mposolda at redhat.com (Marek Posolda) Date: Wed, 4 Nov 2015 08:26:46 +0100 Subject: [keycloak-dev] Plan for "First login with identity brokers" In-Reply-To: <5638C654.6060703@redhat.com> References: <562F36E8.9060501@redhat.com> <562F7693.80805@redhat.com> <562F9286.5020205@redhat.com> <563130DB.2090604@redhat.com> <5631E9FA.1050302@redhat.com> <5632211D.5020706@redhat.com> <56338541.5060403@redhat.com> <5638C654.6060703@redhat.com> Message-ID: <5639B336.3060004@redhat.com> On 03/11/15 15:36, Bill Burke wrote: > I hope the argument here is just what the default should be. > > IMO, the default should be that a new account per social provider is > created and email duplicates are allowed. User would manually merge > accounts via the account service panel if they want. -1 for allow email duplicates. If you login through Facebook and there is already user with email mposolda at gmail.com how you would create new account? Save the email into some different attribute "email_no2" ? Isn't it just ugly workaround to bypass our unique constraint for emails? Then if user wants to login in username/password form and he fills mposolda at gmail.com in login form, which account it should use? The first one or the second one? TBH allowing email duplications would be confusing for users. They will mess the accounts without knowing which of their account are they currently using. In classic registration form, we don't allow email duplications and we just display error message "Email already exists". User needs to use different email or restore his first account ( via "Forget password" or contacting admin etc). Not sure why this behaviour should be different for registration via identity provider. > > These features are equally important IMO: > * Broker providers should be able to be automatically trusted with a > switch and automatically merge accounts. > * brokers should have a flow attached to them so that they can require > account merging and such. Merging 2 existing users in account management would be nice to have. Question is if we really needs it as it is a bit complex thing and nobody really quested it so far AFAIK? Anyway some idea how it could work: * User is logged in account management as "user2" in "Federated identities" tab. He clicks to "Link my account with Twitter" * He login to twitter, but there is already existing Keycloak user "user1" linked to this Twitter account * Account management will display message "There is already existing user user1 linked to Twitter. Do you want to merge accounts?" * User clicks "yes" . There might be some additional verification configured in the flow (for example email confirmation). But not sure if it's critical as user is already able to login as "user1" via Twitter, so he is already proved to be owner of both accounts. * Now the merging itself can work like: -- user1 will have flag MERGED on his account with the attribute "parent_user" pointing to user2 . -- user2 will be added to all roles and groups, which user1 had. So defacto user2 will have union of roles+groups of both users. -- All attributes of user1 will be merged to user2 . If there is attribute "foo" available on both user and there are conflicting user, the original value of user2 will be kept. -- User2 will be added to all federated identities of user1. Again, if there is conflict (For example if both users have different Facebook identity), the original user2 federated identity has preference. -- Account "user1" will become decomissioned and it's not possible anymore to login into it. When user later authenticates as user1 with his user1 password, he will be authenticated as user2 instead. WDYT? Should I go this way? Marek From erik.mulder at docdatapayments.com Wed Nov 4 06:03:59 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Wed, 4 Nov 2015 12:03:59 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> Hi everybody, Quick intro: I'm part of a development team in The Netherlands that is building a company-wide SSO solution. We've chosen KeyCloak to realize this and will use OpenID Connect to secure our REST services. It's a great product and seems to be the only one having both support for all kinds of security standards and a model and GUI for users and roles. Thanks for creating it! :) (if this should be asked instead on the users mailing list, please correct me and I'll post it there) So far, so good, but we have some extra requirements that do not fit into the base KeyCloak data model. See below for details if you're interested. My question is: what is the preferred way / best practice to extend the functionality of KeyCloak while keeping the impact on the original sources to a minimum? Of course we could just fork the most recent version and start hacking away, but we'd like to be able to upgrade to newer versions of KeyCloak without too much hassle. Possibilities that we've come up with so far: 1. Create completely separate modules that will extend the functionality the way we need. 2. Fork on Github, apply custom changes, and try to merge in updates from the master / release branches / tags 3. Apply custom changes on KeyCloak artifacts using a Maven plugin, such as Truezip (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - manipulate zip files by adding/removing/replacing or Shade (http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple jars to 1 'uber-jar' containing the contents of both and when overlapping decide on conflicts through configuration. Of course number 1 is preferred, but I do not see how to add custom mappers or JPA entities without making changes in the original module files. The other options seem like valid alternatives, but maybe there is better / standard way to do this. So any help / insight / shared experience on this is much appreciated, thanks! Kind regards, Erik Mulder Senior Software Engineer Docdata Payments - NL P.S. Details on why we want to extend the KeyCloak data model: (any feedback on the contents of this P.S. is also welcome!) Our clients are merchants that have several webshops. We manage their online payments (shopping cart checkout). We want to be able to let a merchant manage their own users and let a user have different roles for different webshops within the same merchant. The overall possible roles are fixed though, no specific roles per merchant. We could create a separate realm for every merchant, but then we need to duplicate all roles every time. Furthermore, in KeyCloak there is no concept of a role within a certain context. This is very understandable, since every situation has it's own requirements. We did a proof of concept by adding tables and entities for Merchant, UserMerchant, UserMerchantRole etc. and adding a custom mapper that can put this information on the Access token. Worked like a charm! But it does need some changes in the KeyCloak modules and sources to work, hence the question above. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151104/fb7235a8/attachment.html From psilva at redhat.com Wed Nov 4 06:32:48 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Wed, 4 Nov 2015 06:32:48 -0500 (EST) Subject: [keycloak-dev] roles vs. groups In-Reply-To: <56395BA3.6020509@redhat.com> References: <5639216B.9050500@redhat.com> <1447226686.2954301.1446588117003.JavaMail.zimbra@redhat.com> <56395BA3.6020509@redhat.com> Message-ID: <539688376.3287936.1446636768515.JavaMail.zimbra@redhat.com> > > I'm just proposing a role namespace. i.e. Group A defines a role called > "Manager". Bill has role mapping A.Manager. I want to avoid > overlapping with what you are doing. Each group would have some built > in declared roles: i.e. add-user, manage-user, manage-group, etc...So > that a user can be permitted to manage the members of the group. A few > people have been asking for this. > So my understanding is correct. This is what I meant by "GroupRole" relationships, which is also supported in PL. However, there we don't have roles within a group. But you can just assign any existing role to an user in the scope of a specific group. Regarding the overlap, there is no overlap with what I'm doing. What I'm doing is permission, what an user is actually allowed to do, considering the resources and the actions that can be performed on them (plus other things related authz). Like I said, we are going to use all the stuff you are doing to actually define permissions/policies for protected resources. Maybe the names "Permission" and "Rights" are not the best ones for what you are doing ? Maybe you should keep "Role Mapping" instead ? Conceptually, what you are doing is not permissions ... From gerbermichi at me.com Wed Nov 4 07:24:18 2015 From: gerbermichi at me.com (Michael Gerber) Date: Wed, 04 Nov 2015 12:24:18 +0000 (GMT) Subject: [keycloak-dev] username guessing Message-ID: @Bill What do you think about this? Do you prefere the "new way" or the old one? Am 29. Oktober 2015 um 07:15 schrieb Michael Gerber : You showed in the passt the correct error message only if the user has entered the correct password. In other words, you can split the userValidation into a pre and post validation, so you have the possibility to show sensitive messages only to authenticated users. Am 29.10.2015 um 00:42 schrieb Bill Burke : Hmmm...IIRC I kept that there because, if the account is disabled how would the user ever know? This is even more important with a temporarily disabled account. On 10/28/2015 5:48 PM, Michael Gerber wrote: Just create a new user, disable it and try to log in with the username and a wrong password. And you will get the following error message: Account is disabled, contact admin. On 28.10.2015, at 20:50, Bill Burke wrote: How is this possible? On 10/28/2015 10:53 AM, Michael Gerber wrote: Hi all, it is possible to guess the username of disabled users. This was not possible in earlier versions of keycloak. Is this on purpose? Best Michael _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151104/9a1582ed/attachment-0001.html From psilva at redhat.com Wed Nov 4 07:46:41 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Wed, 4 Nov 2015 07:46:41 -0500 (EST) Subject: [keycloak-dev] roles vs. groups In-Reply-To: References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> Message-ID: <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> > > Let's supose, you have a group called "GroupA", that group have roles > "Create invoice" this has 13 permisions, "Remove invoice" this has 5 > permisions, "Update invoice" this has 19 permisions... > > I asign 25 users to Group A, but 2 users, should not have 3 permisions > that are different in both users, should I need to create "GroupB" and > "GroupC" with the exact permissions, just to handle this 3 permisions > exclusions? > > It probably can be a little overkill, but IHMO is more flexible than > an all or nothing approach. > I see your point. However, IMO, when you start to use negatives you may end up with something more complicate and hard to follow. What I meant by positive logic is not an all or nothing approach, but use positives permissions and play with them to match your requirements. For instance, we are developing some additional authorization features to Keycloak. There you can define a set of policies that rule access to your resources based on different type of access control mechanisms: rbac, contextual/risk-based, abac, user-based, etc. Policies can be applied to a resource type, resource instance, a specific scope (eg.: create, remove, delete, etc) or a combination of them. Policies are also independent of each other and can be reused and combined in different ways to actually define and enforce authorization for a given resource/scope. Considering you scenario, you would have something like that: - Only Group A Policy (defines that all members of Group A can *do something*) - Resource X Policy (here we are telling what Group A can do. In this case, access Resource X and any of its scopes) - Applies Only Group A Policy You can also have something like: - Only Group A Policy (defines that all members of Group A can *do something*) - Create Invoice Policy (here we're telling that Group A can access a specific scope.) - Applies Only Group A Policy Now, if you want specific policies for some of your users, you can do something like: - Exceptional Y Policy (which can define a specific user, role, group, any contextual information like current time or authentication method) - Create Invoice Policy - Applies Only Group A Policy - Applies Exceptional Y Policy That will provide flexible policies, make them more meaningful and easy to follow/understand. We are considering protect resources as important assets within an organization, which should be ruled by a set of policies that must be applied before accessing them. I would appreciate if you could provide some more info about your use case. Right now I'm writing some examples and would be nice to check how your requirements can be addressed (or not :)) with what we did so far. Thanks. Pedro Igor From ssilvert at redhat.com Wed Nov 4 08:43:52 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Wed, 04 Nov 2015 08:43:52 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> Message-ID: <563A0B98.4080909@redhat.com> First, +1 to adding groups and permissions, because that's what our users expect to see. I'm going to go ahead and say what I was hoping to say last night. It's hard to visualize this stuff without seeing diagrams, but I'm wondering if we still need composites. That is, once we have groups, roles, and permissions, do composites really solve something that can't be done as easily or better with the groups, roles, and permissions? Composites have been a point of confusion. I can remember when I first started working with Keycloak that it took awhile to get comfortable with composites. I think that's the reaction our users are having as well. From a technical standpoint, they are very powerful. But that's not readily apparent to a lot of people, which is why users are always asking for groups. Perhaps we should give up some technical elegance for something that's easier to understand. If composites are only there for convenience, I'd say get rid of them. IMO, we should favor something that's easy to understand and still gets the job done. (As a bonus, simpler data structures are easier to deal with as we add new features!) On 11/4/2015 7:46 AM, Pedro Igor Silva wrote: >> Let's supose, you have a group called "GroupA", that group have roles >> "Create invoice" this has 13 permisions, "Remove invoice" this has 5 >> permisions, "Update invoice" this has 19 permisions... >> >> I asign 25 users to Group A, but 2 users, should not have 3 permisions >> that are different in both users, should I need to create "GroupB" and >> "GroupC" with the exact permissions, just to handle this 3 permisions >> exclusions? >> >> It probably can be a little overkill, but IHMO is more flexible than >> an all or nothing approach. >> > I see your point. However, IMO, when you start to use negatives you may end up with something more complicate and hard to follow. > > What I meant by positive logic is not an all or nothing approach, but use positives permissions and play with them to match your requirements. > > For instance, we are developing some additional authorization features to Keycloak. There you can define a set of policies that rule access to your resources based on different type of access control mechanisms: rbac, contextual/risk-based, abac, user-based, etc. > > Policies can be applied to a resource type, resource instance, a specific scope (eg.: create, remove, delete, etc) or a combination of them. Policies are also independent of each other and can be reused and combined in different ways to actually define and enforce authorization for a given resource/scope. > > Considering you scenario, you would have something like that: > > - Only Group A Policy (defines that all members of Group A can *do something*) > - Resource X Policy (here we are telling what Group A can do. In this case, access Resource X and any of its scopes) > - Applies Only Group A Policy > > You can also have something like: > > - Only Group A Policy (defines that all members of Group A can *do something*) > - Create Invoice Policy (here we're telling that Group A can access a specific scope.) > - Applies Only Group A Policy > > Now, if you want specific policies for some of your users, you can do something like: > > - Exceptional Y Policy (which can define a specific user, role, group, any contextual information like current time or authentication method) > - Create Invoice Policy > - Applies Only Group A Policy > - Applies Exceptional Y Policy > > That will provide flexible policies, make them more meaningful and easy to follow/understand. We are considering protect resources as important assets within an organization, which should be ruled by a set of policies that must be applied before accessing them. > > I would appreciate if you could provide some more info about your use case. Right now I'm writing some examples and would be nice to check how your requirements can be addressed (or not :)) with what we did so far. > > Thanks. > Pedro Igor > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev From jorsol at gmail.com Wed Nov 4 08:56:36 2015 From: jorsol at gmail.com (=?UTF-8?Q?Jorge_Sol=C3=B3rzano?=) Date: Wed, 4 Nov 2015 07:56:36 -0600 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> Message-ID: Well, based on what you said on policies looks fine to me... Give a look to the following scenario and if it can be acomplish with KC. The organization has many subdivisions like this: Org |------Subsidiary 1 | |------Branch office 1.1 | | |------Office 1.1.1 | | |------Office 1.1.2 | |------Branch office 1.2 | |------Office 1.2.1 |------Subsidiary 2 | |------Branch office 2.1 | |------Office 2.2.1 | |------Office 2.2.2 ... Can I define authorization in a way that Office 2.2.2 can only touch the resources specific to it, Branch office 1.1 can see the resources of Office 1.1.2 and Office 1.1.1, Subsidiary 1 can see all in Branch office 1.1 and 1.2... and so on... Jorge Sol?rzano On Wed, Nov 4, 2015 at 6:46 AM, Pedro Igor Silva wrote: >> >> Let's supose, you have a group called "GroupA", that group have roles >> "Create invoice" this has 13 permisions, "Remove invoice" this has 5 >> permisions, "Update invoice" this has 19 permisions... >> >> I asign 25 users to Group A, but 2 users, should not have 3 permisions >> that are different in both users, should I need to create "GroupB" and >> "GroupC" with the exact permissions, just to handle this 3 permisions >> exclusions? >> >> It probably can be a little overkill, but IHMO is more flexible than >> an all or nothing approach. >> > > I see your point. However, IMO, when you start to use negatives you may end up with something more complicate and hard to follow. > > What I meant by positive logic is not an all or nothing approach, but use positives permissions and play with them to match your requirements. > > For instance, we are developing some additional authorization features to Keycloak. There you can define a set of policies that rule access to your resources based on different type of access control mechanisms: rbac, contextual/risk-based, abac, user-based, etc. > > Policies can be applied to a resource type, resource instance, a specific scope (eg.: create, remove, delete, etc) or a combination of them. Policies are also independent of each other and can be reused and combined in different ways to actually define and enforce authorization for a given resource/scope. > > Considering you scenario, you would have something like that: > > - Only Group A Policy (defines that all members of Group A can *do something*) > - Resource X Policy (here we are telling what Group A can do. In this case, access Resource X and any of its scopes) > - Applies Only Group A Policy > > You can also have something like: > > - Only Group A Policy (defines that all members of Group A can *do something*) > - Create Invoice Policy (here we're telling that Group A can access a specific scope.) > - Applies Only Group A Policy > > Now, if you want specific policies for some of your users, you can do something like: > > - Exceptional Y Policy (which can define a specific user, role, group, any contextual information like current time or authentication method) > - Create Invoice Policy > - Applies Only Group A Policy > - Applies Exceptional Y Policy > > That will provide flexible policies, make them more meaningful and easy to follow/understand. We are considering protect resources as important assets within an organization, which should be ruled by a set of policies that must be applied before accessing them. > > I would appreciate if you could provide some more info about your use case. Right now I'm writing some examples and would be nice to check how your requirements can be addressed (or not :)) with what we did so far. > > Thanks. > Pedro Igor > > > From bburke at redhat.com Wed Nov 4 09:03:24 2015 From: bburke at redhat.com (Bill Burke) Date: Wed, 4 Nov 2015 09:03:24 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <539688376.3287936.1446636768515.JavaMail.zimbra@redhat.com> References: <5639216B.9050500@redhat.com> <1447226686.2954301.1446588117003.JavaMail.zimbra@redhat.com> <56395BA3.6020509@redhat.com> <539688376.3287936.1446636768515.JavaMail.zimbra@redhat.com> Message-ID: <563A102C.8070701@redhat.com> On 11/4/2015 6:32 AM, Pedro Igor Silva wrote: >> >> I'm just proposing a role namespace. i.e. Group A defines a role called >> "Manager". Bill has role mapping A.Manager. I want to avoid >> overlapping with what you are doing. Each group would have some built >> in declared roles: i.e. add-user, manage-user, manage-group, etc...So >> that a user can be permitted to manage the members of the group. A few >> people have been asking for this. >> > > So my understanding is correct. This is what I meant by "GroupRole" relationships, which is also supported in PL. However, there we don't have roles within a group. But you can just assign any existing role to an user in the scope of a specific group. > What do you mean by "assign any existing role to an user in the scope of a specific group"? User has Manager role for Group A? So, the relationship (or join table) is User->Role->Group? User would not have Manager role for Group B. That relationship would have to be set up separately. Isn't that different than "all users of Group A have the manager role"? The difference would be in the token creation. The former, "User has Manage role for Group A" would have to look like this: "groups": [ { "name" : "Group A", "roles" : [ "Manager" ] } ] The latter "users of Group A have the manager role", would just add "manager" to the list of roles the user has: "roles: ["Manager"] There really is no equivalent in Java EE for leveraging a User/Role/Group relationship. We'd have to add it to our adapters. Isn't "users of Group A have the manager role" enough? > Regarding the overlap, there is no overlap with what I'm doing. What I'm doing is permission, what an user is actually allowed to do, considering the resources and the actions that can be performed on them (plus other things related authz). Like I said, we are going to use all the stuff you are doing to actually define permissions/policies for protected resources. > > Maybe the names "Permission" and "Rights" are not the best ones for what you are doing ? Maybe you should keep "Role Mapping" instead ? Conceptually, what you are doing is not permissions ... > So Role Mappings instead of Permissions for Groups. I need a better name for "composite role". I also need a better name for "Role Namespace". -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Wed Nov 4 09:15:02 2015 From: bburke at redhat.com (Bill Burke) Date: Wed, 4 Nov 2015 09:15:02 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <563A0B98.4080909@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> <563A0B98.4080909@redhat.com> Message-ID: <563A12E6.2040206@redhat.com> I've alread stated the reason for composite roles: Say you have a set of applications under the Sales and Marketing Department: A Leads Application, Eloqua, and Salesforce. Each of the applications has a set of roles that are used to manage access to various features of each application. For example, each app might have an "admin" role. You would then want to organize permissions into categories and assign coarser grain roles to individual users. So, you would create a "Sales Admin" composite role that contains the "admin" role of each sales application. Composite roles allow you to group together roles into role catagories that you can assign to a specific user or user group. User Groups are different as you want to assign a set of permissions to a group of users. So composite roles are used to group together roles of a set of applications. User Groups are used to grant a set of perissions to a set of users. On 11/4/2015 8:43 AM, Stan Silvert wrote: > First, +1 to adding groups and permissions, because that's what our > users expect to see. > > I'm going to go ahead and say what I was hoping to say last night. It's > hard to visualize this stuff without seeing diagrams, but I'm wondering > if we still need composites. That is, once we have groups, roles, and > permissions, do composites really solve something that can't be done as > easily or better with the groups, roles, and permissions? > > Composites have been a point of confusion. I can remember when I first > started working with Keycloak that it took awhile to get comfortable > with composites. I think that's the reaction our users are having as > well. From a technical standpoint, they are very powerful. But that's > not readily apparent to a lot of people, which is why users are always > asking for groups. Perhaps we should give up some technical elegance > for something that's easier to understand. > > If composites are only there for convenience, I'd say get rid of them. > IMO, we should favor something that's easy to understand and still gets > the job done. (As a bonus, simpler data structures are easier to deal > with as we add new features!) > > On 11/4/2015 7:46 AM, Pedro Igor Silva wrote: >>> Let's supose, you have a group called "GroupA", that group have roles >>> "Create invoice" this has 13 permisions, "Remove invoice" this has 5 >>> permisions, "Update invoice" this has 19 permisions... >>> >>> I asign 25 users to Group A, but 2 users, should not have 3 permisions >>> that are different in both users, should I need to create "GroupB" and >>> "GroupC" with the exact permissions, just to handle this 3 permisions >>> exclusions? >>> >>> It probably can be a little overkill, but IHMO is more flexible than >>> an all or nothing approach. >>> >> I see your point. However, IMO, when you start to use negatives you may end up with something more complicate and hard to follow. >> >> What I meant by positive logic is not an all or nothing approach, but use positives permissions and play with them to match your requirements. >> >> For instance, we are developing some additional authorization features to Keycloak. There you can define a set of policies that rule access to your resources based on different type of access control mechanisms: rbac, contextual/risk-based, abac, user-based, etc. >> >> Policies can be applied to a resource type, resource instance, a specific scope (eg.: create, remove, delete, etc) or a combination of them. Policies are also independent of each other and can be reused and combined in different ways to actually define and enforce authorization for a given resource/scope. >> >> Considering you scenario, you would have something like that: >> >> - Only Group A Policy (defines that all members of Group A can *do something*) >> - Resource X Policy (here we are telling what Group A can do. In this case, access Resource X and any of its scopes) >> - Applies Only Group A Policy >> >> You can also have something like: >> >> - Only Group A Policy (defines that all members of Group A can *do something*) >> - Create Invoice Policy (here we're telling that Group A can access a specific scope.) >> - Applies Only Group A Policy >> >> Now, if you want specific policies for some of your users, you can do something like: >> >> - Exceptional Y Policy (which can define a specific user, role, group, any contextual information like current time or authentication method) >> - Create Invoice Policy >> - Applies Only Group A Policy >> - Applies Exceptional Y Policy >> >> That will provide flexible policies, make them more meaningful and easy to follow/understand. We are considering protect resources as important assets within an organization, which should be ruled by a set of policies that must be applied before accessing them. >> >> I would appreciate if you could provide some more info about your use case. Right now I'm writing some examples and would be nice to check how your requirements can be addressed (or not :)) with what we did so far. >> >> Thanks. >> Pedro Igor >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Wed Nov 4 09:30:34 2015 From: bburke at redhat.com (Bill Burke) Date: Wed, 4 Nov 2015 09:30:34 -0500 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes In-Reply-To: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> Message-ID: <563A168A.2010402@redhat.com> Custom mappers should be possible. I didn't document it as I wasn't sure if we wanted to make the SPI public. Custom mappers should just follow the Provider SPI and they will be picked up. If you see the META-INF/services/... file in the resources directory of the "services" or "broker" modules you'll see how to set this up. As for extending the JPA datamodel, what you could do is write a new JPA Connections Provider and plug that in. See connections/jpa. I'm not sure how you would handle the liquibase db migration. On 11/4/2015 6:03 AM, Erik Mulder wrote: > Hi everybody, > > Quick intro: I?m part of a development team in The Netherlands that is > building a company-wide SSO solution. We?ve chosen KeyCloak to realize > this and will use OpenID Connect to secure our REST services. It?s a > great product and seems to be the only one having both support for all > kinds of security standards and a model and GUI for users and roles. > Thanks for creating it! J > > (if this should be asked instead on the users mailing list, please > correct me and I?ll post it there) > > So far, so good, but we have some extra requirements that do not fit > into the base KeyCloak data model. See below for details if you?re > interested. My question is: what is the preferred way / best practice to > extend the functionality of KeyCloak while keeping the impact on the > original sources to a minimum? Of course we could just fork the most > recent version and start hacking away, but we?d like to be able to > upgrade to newer versions of KeyCloak without too much hassle. > Possibilities that we?ve come up with so far: > > 1.Create completely separate modules that will extend the functionality > the way we need. > > 2.Fork on Github, apply custom changes, and try to merge in updates from > the master / release branches / tags > > 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such > as Truezip > (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - > manipulate zip files by adding/removing/replacing or Shade > (http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple > jars to 1 'uber-jar' containing the contents of both and when > overlapping decide on conflicts through configuration. > > Of course number 1 is preferred, but I do not see how to add custom > mappers or JPA entities without making changes in the original module > files. The other options seem like valid alternatives, but maybe there > is better / standard way to do this. So any help / insight / shared > experience on this is much appreciated, thanks! > > Kind regards, > > Erik Mulder > > Senior Software Engineer > > Docdata Payments ? NL > > P.S. Details on why we want to extend the KeyCloak data model: (any > feedback on the contents of this P.S. is also welcome!) > > Our clients are merchants that have several webshops. We manage their > online payments (shopping cart checkout). We want to be able to let a > merchant manage their own users and let a user have different roles for > different webshops within the same merchant. The overall possible roles > are fixed though, no specific roles per merchant. We could create a > separate realm for every merchant, but then we need to duplicate all > roles every time. Furthermore, in KeyCloak there is no concept of a role > within a certain context. This is very understandable, since every > situation has it?s own requirements. We did a proof of concept by adding > tables and entities for Merchant, UserMerchant, UserMerchantRole etc. > and adding a custom mapper that can put this information on the Access > token. Worked like a charm! But it does need some changes in the > KeyCloak modules and sources to work, hence the question above. > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From psilva at redhat.com Wed Nov 4 09:36:27 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Wed, 4 Nov 2015 09:36:27 -0500 (EST) Subject: [keycloak-dev] roles vs. groups In-Reply-To: <563A102C.8070701@redhat.com> References: <5639216B.9050500@redhat.com> <1447226686.2954301.1446588117003.JavaMail.zimbra@redhat.com> <56395BA3.6020509@redhat.com> <539688376.3287936.1446636768515.JavaMail.zimbra@redhat.com> <563A102C.8070701@redhat.com> Message-ID: <412784568.3422856.1446647787493.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Bill Burke" > To: "Pedro Igor Silva" > Cc: keycloak-dev at lists.jboss.org > Sent: Wednesday, November 4, 2015 12:03:24 PM > Subject: Re: [keycloak-dev] roles vs. groups > > > > On 11/4/2015 6:32 AM, Pedro Igor Silva wrote: > >> > >> I'm just proposing a role namespace. i.e. Group A defines a role called > >> "Manager". Bill has role mapping A.Manager. I want to avoid > >> overlapping with what you are doing. Each group would have some built > >> in declared roles: i.e. add-user, manage-user, manage-group, etc...So > >> that a user can be permitted to manage the members of the group. A few > >> people have been asking for this. > >> > > > > So my understanding is correct. This is what I meant by "GroupRole" > > relationships, which is also supported in PL. However, there we don't have > > roles within a group. But you can just assign any existing role to an user > > in the scope of a specific group. > > > > What do you mean by "assign any existing role to an user in the scope of > a specific group"? User has Manager role for Group A? So, the > relationship (or join table) is User->Role->Group? User would not have > Manager role for Group B. That relationship would have to be set up > separately. Yes, that is what I meant: "User has Manager role for Group A". And yes, User would not have Manager role for Group B, only Group A. > > Isn't that different than "all users of Group A have the manager role"? Yeah, that is different. The case above is about inheriting roles from the group you are member of, right ? > > The difference would be in the token creation. The former, "User has > Manage role for Group A" would have to look like this: > > "groups": [ > { "name" : "Group A", > "roles" : [ "Manager" ] > } > ] +1 > > The latter "users of Group A have the manager role", would just add > "manager" to the list of roles the user has: > > "roles: ["Manager"] +1 > > > There really is no equivalent in Java EE for leveraging a > User/Role/Group relationship. We'd have to add it to our adapters. > Isn't "users of Group A have the manager role" enough? For most cases yes. But if you want to be more flexible you can support that as well. IIRC, Hawkular wants group role. And it might be useful for others as well. > > > Regarding the overlap, there is no overlap with what I'm doing. What I'm > > doing is permission, what an user is actually allowed to do, considering > > the resources and the actions that can be performed on them (plus other > > things related authz). Like I said, we are going to use all the stuff you > > are doing to actually define permissions/policies for protected resources. > > > > Maybe the names "Permission" and "Rights" are not the best ones for what > > you are doing ? Maybe you should keep "Role Mapping" instead ? > > Conceptually, what you are doing is not permissions ... > > > > So Role Mappings instead of Permissions for Groups. I need a better > name for "composite role". I also need a better name for "Role Namespace". > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > From ssilvert at redhat.com Wed Nov 4 10:26:13 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Wed, 04 Nov 2015 10:26:13 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <563A12E6.2040206@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> <563A0B98.4080909@redhat.com> <563A12E6.2040206@redhat.com> Message-ID: <563A2395.8010900@redhat.com> On 11/4/2015 9:15 AM, Bill Burke wrote: > I've alread stated the reason for composite roles: > > Say you have a set of applications under the Sales and Marketing > Department: A Leads Application, Eloqua, and Salesforce. Each of the > applications has a set of roles that are used to manage access to > various features of each application. For example, each app might have > an "admin" role. You would then want to organize permissions into > categories and assign coarser grain roles to individual users. So, you > would create a "Sales Admin" composite role that contains the "admin" > role of each sales application. Composite roles allow you to group > together roles into role catagories that you can assign to a specific > user or user group. > > User Groups are different as you want to assign a set of permissions to > a group of users. > > So composite roles are used to group together roles of a set of > applications. User Groups are used to grant a set of perissions to a > set of users. Maybe it's just me, but I think of user groups as just a way to group users, and roles as a way to group permissions. Roles are assigned to user groups. Permissions are assigned to roles. I don't see why you need anything more. In your example, each application has an admin role that has a set of permissions for the application. Each admin role can be assigned to a Sales Admin user group. Sales Admin users are assigned to the Sales Admin user group. Done. User Groups, Roles, and Permissions are common concepts. Are there other systems that use composite roles? I honestly don't know the answer, but I'm pressing on this point right now because I don't want us to be stuck with a concept that users don't readily understand and has not been found to be useful elsewhere. > > On 11/4/2015 8:43 AM, Stan Silvert wrote: >> First, +1 to adding groups and permissions, because that's what our >> users expect to see. >> >> I'm going to go ahead and say what I was hoping to say last night. It's >> hard to visualize this stuff without seeing diagrams, but I'm wondering >> if we still need composites. That is, once we have groups, roles, and >> permissions, do composites really solve something that can't be done as >> easily or better with the groups, roles, and permissions? >> >> Composites have been a point of confusion. I can remember when I first >> started working with Keycloak that it took awhile to get comfortable >> with composites. I think that's the reaction our users are having as >> well. From a technical standpoint, they are very powerful. But that's >> not readily apparent to a lot of people, which is why users are always >> asking for groups. Perhaps we should give up some technical elegance >> for something that's easier to understand. >> >> If composites are only there for convenience, I'd say get rid of them. >> IMO, we should favor something that's easy to understand and still gets >> the job done. (As a bonus, simpler data structures are easier to deal >> with as we add new features!) >> >> On 11/4/2015 7:46 AM, Pedro Igor Silva wrote: >>>> Let's supose, you have a group called "GroupA", that group have roles >>>> "Create invoice" this has 13 permisions, "Remove invoice" this has 5 >>>> permisions, "Update invoice" this has 19 permisions... >>>> >>>> I asign 25 users to Group A, but 2 users, should not have 3 permisions >>>> that are different in both users, should I need to create "GroupB" and >>>> "GroupC" with the exact permissions, just to handle this 3 permisions >>>> exclusions? >>>> >>>> It probably can be a little overkill, but IHMO is more flexible than >>>> an all or nothing approach. >>>> >>> I see your point. However, IMO, when you start to use negatives you may end up with something more complicate and hard to follow. >>> >>> What I meant by positive logic is not an all or nothing approach, but use positives permissions and play with them to match your requirements. >>> >>> For instance, we are developing some additional authorization features to Keycloak. There you can define a set of policies that rule access to your resources based on different type of access control mechanisms: rbac, contextual/risk-based, abac, user-based, etc. >>> >>> Policies can be applied to a resource type, resource instance, a specific scope (eg.: create, remove, delete, etc) or a combination of them. Policies are also independent of each other and can be reused and combined in different ways to actually define and enforce authorization for a given resource/scope. >>> >>> Considering you scenario, you would have something like that: >>> >>> - Only Group A Policy (defines that all members of Group A can *do something*) >>> - Resource X Policy (here we are telling what Group A can do. In this case, access Resource X and any of its scopes) >>> - Applies Only Group A Policy >>> >>> You can also have something like: >>> >>> - Only Group A Policy (defines that all members of Group A can *do something*) >>> - Create Invoice Policy (here we're telling that Group A can access a specific scope.) >>> - Applies Only Group A Policy >>> >>> Now, if you want specific policies for some of your users, you can do something like: >>> >>> - Exceptional Y Policy (which can define a specific user, role, group, any contextual information like current time or authentication method) >>> - Create Invoice Policy >>> - Applies Only Group A Policy >>> - Applies Exceptional Y Policy >>> >>> That will provide flexible policies, make them more meaningful and easy to follow/understand. We are considering protect resources as important assets within an organization, which should be ruled by a set of policies that must be applied before accessing them. >>> >>> I would appreciate if you could provide some more info about your use case. Right now I'm writing some examples and would be nice to check how your requirements can be addressed (or not :)) with what we did so far. >>> >>> Thanks. >>> Pedro Igor >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> From bburke at redhat.com Wed Nov 4 10:37:59 2015 From: bburke at redhat.com (Bill Burke) Date: Wed, 4 Nov 2015 10:37:59 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <563A2395.8010900@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> <563A0B98.4080909@redhat.com> <563A12E6.2040206@redhat.com> <563A2395.8010900@redhat.com> Message-ID: <563A2657.1020209@redhat.com> On 11/4/2015 10:26 AM, Stan Silvert wrote: > On 11/4/2015 9:15 AM, Bill Burke wrote: >> I've alread stated the reason for composite roles: >> >> Say you have a set of applications under the Sales and Marketing >> Department: A Leads Application, Eloqua, and Salesforce. Each of the >> applications has a set of roles that are used to manage access to >> various features of each application. For example, each app might have >> an "admin" role. You would then want to organize permissions into >> categories and assign coarser grain roles to individual users. So, you >> would create a "Sales Admin" composite role that contains the "admin" >> role of each sales application. Composite roles allow you to group >> together roles into role catagories that you can assign to a specific >> user or user group. >> >> User Groups are different as you want to assign a set of permissions to >> a group of users. >> >> So composite roles are used to group together roles of a set of >> applications. User Groups are used to grant a set of perissions to a >> set of users. > Maybe it's just me, but I think of user groups as just a way to group > users, and roles as a way to group permissions. Roles are assigned to > user groups. Permissions are assigned to roles. > We dont' have the concept of a permission, so, assigning roles to a composite role is equivalent right now of assigning permissions to a role. > I don't see why you need anything more. In your example, each > application has an admin role that has a set of permissions for the > application. Each admin role can be assigned to a Sales Admin user > group. Sales Admin users are assigned to the Sales Admin user group. > Done. > App developers focus on designing the roles/permission model for the applications and would deal with roles, composite roles, and clients. User admins would focus on managing users and defining groups and assigning permissions/roles to groups and users. Instead of dealing with fine-grain roles/permissions for each and every application, user admins just deal with coarse grain composite roles. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From psilva at redhat.com Wed Nov 4 11:01:02 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Wed, 4 Nov 2015 11:01:02 -0500 (EST) Subject: [keycloak-dev] roles vs. groups In-Reply-To: References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> Message-ID: <256635371.3576177.1446652862909.JavaMail.zimbra@redhat.com> Yeah, it should be possible. Once Bill finishes his group stuff we can support that. Right now, the Authz Server we're working on does not provide group-based policies, which is a requirement to get your use case addressed. But once groups are there, you can easily define a policy that let's you associate a set of resources to a group and enforce authorization. We can even provide options to this type of policy like: - An option to enable/disable roles inheritance. In case you don't want to inherit roles when checking for permissions. - An option do enable/disable group hierarchy checks. In case you want to match only a specific group. I'll keep your scenario in mind and test with the Authz Server once we have groups in. Thanks. ----- Original Message ----- From: "Jorge Sol?rzano" To: "Pedro Igor Silva" Cc: "Bill Burke" , keycloak-dev at lists.jboss.org Sent: Wednesday, November 4, 2015 11:56:36 AM Subject: Re: [keycloak-dev] roles vs. groups Well, based on what you said on policies looks fine to me... Give a look to the following scenario and if it can be acomplish with KC. The organization has many subdivisions like this: Org |------Subsidiary 1 | |------Branch office 1.1 | | |------Office 1.1.1 | | |------Office 1.1.2 | |------Branch office 1.2 | |------Office 1.2.1 |------Subsidiary 2 | |------Branch office 2.1 | |------Office 2.2.1 | |------Office 2.2.2 ... Can I define authorization in a way that Office 2.2.2 can only touch the resources specific to it, Branch office 1.1 can see the resources of Office 1.1.2 and Office 1.1.1, Subsidiary 1 can see all in Branch office 1.1 and 1.2... and so on... Jorge Sol?rzano On Wed, Nov 4, 2015 at 6:46 AM, Pedro Igor Silva wrote: >> >> Let's supose, you have a group called "GroupA", that group have roles >> "Create invoice" this has 13 permisions, "Remove invoice" this has 5 >> permisions, "Update invoice" this has 19 permisions... >> >> I asign 25 users to Group A, but 2 users, should not have 3 permisions >> that are different in both users, should I need to create "GroupB" and >> "GroupC" with the exact permissions, just to handle this 3 permisions >> exclusions? >> >> It probably can be a little overkill, but IHMO is more flexible than >> an all or nothing approach. >> > > I see your point. However, IMO, when you start to use negatives you may end up with something more complicate and hard to follow. > > What I meant by positive logic is not an all or nothing approach, but use positives permissions and play with them to match your requirements. > > For instance, we are developing some additional authorization features to Keycloak. There you can define a set of policies that rule access to your resources based on different type of access control mechanisms: rbac, contextual/risk-based, abac, user-based, etc. > > Policies can be applied to a resource type, resource instance, a specific scope (eg.: create, remove, delete, etc) or a combination of them. Policies are also independent of each other and can be reused and combined in different ways to actually define and enforce authorization for a given resource/scope. > > Considering you scenario, you would have something like that: > > - Only Group A Policy (defines that all members of Group A can *do something*) > - Resource X Policy (here we are telling what Group A can do. In this case, access Resource X and any of its scopes) > - Applies Only Group A Policy > > You can also have something like: > > - Only Group A Policy (defines that all members of Group A can *do something*) > - Create Invoice Policy (here we're telling that Group A can access a specific scope.) > - Applies Only Group A Policy > > Now, if you want specific policies for some of your users, you can do something like: > > - Exceptional Y Policy (which can define a specific user, role, group, any contextual information like current time or authentication method) > - Create Invoice Policy > - Applies Only Group A Policy > - Applies Exceptional Y Policy > > That will provide flexible policies, make them more meaningful and easy to follow/understand. We are considering protect resources as important assets within an organization, which should be ruled by a set of policies that must be applied before accessing them. > > I would appreciate if you could provide some more info about your use case. Right now I'm writing some examples and would be nice to check how your requirements can be addressed (or not :)) with what we did so far. > > Thanks. > Pedro Igor > > > From erik.mulder at docdatapayments.com Wed Nov 4 11:16:37 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Wed, 4 Nov 2015 17:16:37 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> Thanks for your response! Indeed we already did a proof of concept where we added a custom mapper the way you described (didn't know it was 'protected' territory :). The question is: do we have to override the file 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper in the original project or is there another way where we don't need to touch the original sources and keep all our changes in a separate project? And how can we do it such that it stays easy to upgrade to newer KeyCloak releases? As for JPA: it would be easier to integrate with the existing JPA project. Again we are wondering whether to start modifying original sources (like persistence.xml) or try to 'externalize' our changes somehow and integrate them using existing 'hooks' in the system or maybe merge projects during build. Maybe there is no good answer to this and we'll always be having some manual merge pains when upgrading to new KeyCloak versions. We just wanted to check if there are preferred ways to add functionality with the least amount of impact on the original sources. On 04/11/15 15:30, Bill Burke wrote: > Custom mappers should be possible. I didn't document it as I wasn't > sure if we wanted to make the SPI public. Custom mappers should just > follow the Provider SPI and they will be picked up. If you see the > META-INF/services/... file in the resources directory of the "services" > or "broker" modules you'll see how to set this up. > > As for extending the JPA datamodel, what you could do is write a new JPA > Connections Provider and plug that in. See connections/jpa. I'm not > sure how you would handle the liquibase db migration. > > On 11/4/2015 6:03 AM, Erik Mulder wrote: >> Hi everybody, >> >> Quick intro: I?m part of a development team in The Netherlands that is >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >> this and will use OpenID Connect to secure our REST services. It?s a >> great product and seems to be the only one having both support for all >> kinds of security standards and a model and GUI for users and roles. >> Thanks for creating it! J >> >> (if this should be asked instead on the users mailing list, please >> correct me and I?ll post it there) >> >> So far, so good, but we have some extra requirements that do not fit >> into the base KeyCloak data model. See below for details if you?re >> interested. My question is: what is the preferred way / best practice to >> extend the functionality of KeyCloak while keeping the impact on the >> original sources to a minimum? Of course we could just fork the most >> recent version and start hacking away, but we?d like to be able to >> upgrade to newer versions of KeyCloak without too much hassle. >> Possibilities that we?ve come up with so far: >> >> 1.Create completely separate modules that will extend the functionality >> the way we need. >> >> 2.Fork on Github, apply custom changes, and try to merge in updates from >> the master / release branches / tags >> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >> as Truezip >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >> manipulate zip files by adding/removing/replacing or Shade >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple >> jars to 1 'uber-jar' containing the contents of both and when >> overlapping decide on conflicts through configuration. >> >> Of course number 1 is preferred, but I do not see how to add custom >> mappers or JPA entities without making changes in the original module >> files. The other options seem like valid alternatives, but maybe there >> is better / standard way to do this. So any help / insight / shared >> experience on this is much appreciated, thanks! >> >> Kind regards, >> >> Erik Mulder >> >> Senior Software Engineer >> >> Docdata Payments ? NL >> >> P.S. Details on why we want to extend the KeyCloak data model: (any >> feedback on the contents of this P.S. is also welcome!) >> >> Our clients are merchants that have several webshops. We manage their >> online payments (shopping cart checkout). We want to be able to let a >> merchant manage their own users and let a user have different roles for >> different webshops within the same merchant. The overall possible roles >> are fixed though, no specific roles per merchant. We could create a >> separate realm for every merchant, but then we need to duplicate all >> roles every time. Furthermore, in KeyCloak there is no concept of a role >> within a certain context. This is very understandable, since every >> situation has it?s own requirements. We did a proof of concept by adding >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >> and adding a custom mapper that can put this information on the Access >> token. Worked like a charm! But it does need some changes in the >> KeyCloak modules and sources to work, hence the question above. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> From ssilvert at redhat.com Wed Nov 4 11:21:17 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Wed, 04 Nov 2015 11:21:17 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <563A2657.1020209@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> <563A0B98.4080909@redhat.com> <563A12E6.2040206@redhat.com> <563A2395.8010900@redhat.com> <563A2657.1020209@redhat.com> Message-ID: <563A307D.4070302@redhat.com> On 11/4/2015 10:37 AM, Bill Burke wrote: > > On 11/4/2015 10:26 AM, Stan Silvert wrote: >> On 11/4/2015 9:15 AM, Bill Burke wrote: >>> I've alread stated the reason for composite roles: >>> >>> Say you have a set of applications under the Sales and Marketing >>> Department: A Leads Application, Eloqua, and Salesforce. Each of the >>> applications has a set of roles that are used to manage access to >>> various features of each application. For example, each app might have >>> an "admin" role. You would then want to organize permissions into >>> categories and assign coarser grain roles to individual users. So, you >>> would create a "Sales Admin" composite role that contains the "admin" >>> role of each sales application. Composite roles allow you to group >>> together roles into role catagories that you can assign to a specific >>> user or user group. >>> >>> User Groups are different as you want to assign a set of permissions to >>> a group of users. >>> >>> So composite roles are used to group together roles of a set of >>> applications. User Groups are used to grant a set of perissions to a >>> set of users. >> Maybe it's just me, but I think of user groups as just a way to group >> users, and roles as a way to group permissions. Roles are assigned to >> user groups. Permissions are assigned to roles. >> > We dont' have the concept of a permission, so, assigning roles to a > composite role is equivalent right now of assigning permissions to a role. Isn't that what Pedro is working on right now? > > >> I don't see why you need anything more. In your example, each >> application has an admin role that has a set of permissions for the >> application. Each admin role can be assigned to a Sales Admin user >> group. Sales Admin users are assigned to the Sales Admin user group. >> Done. >> > App developers focus on designing the roles/permission model for the > applications and would deal with roles, composite roles, and clients. > > User admins would focus on managing users and defining groups and > assigning permissions/roles to groups and users. Instead of dealing > with fine-grain roles/permissions for each and every application, user > admins just deal with coarse grain composite roles. > From bburke at redhat.com Wed Nov 4 11:51:56 2015 From: bburke at redhat.com (Bill Burke) Date: Wed, 4 Nov 2015 11:51:56 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <563A307D.4070302@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> <563A0B98.4080909@redhat.com> <563A12E6.2040206@redhat.com> <563A2395.8010900@redhat.com> <563A2657.1020209@redhat.com> <563A307D.4070302@redhat.com> Message-ID: <563A37AC.80604@redhat.com> On 11/4/2015 11:21 AM, Stan Silvert wrote: > On 11/4/2015 10:37 AM, Bill Burke wrote: >> >> On 11/4/2015 10:26 AM, Stan Silvert wrote: >>> On 11/4/2015 9:15 AM, Bill Burke wrote: >>>> I've alread stated the reason for composite roles: >>>> >>>> Say you have a set of applications under the Sales and Marketing >>>> Department: A Leads Application, Eloqua, and Salesforce. Each of the >>>> applications has a set of roles that are used to manage access to >>>> various features of each application. For example, each app might have >>>> an "admin" role. You would then want to organize permissions into >>>> categories and assign coarser grain roles to individual users. So, you >>>> would create a "Sales Admin" composite role that contains the "admin" >>>> role of each sales application. Composite roles allow you to group >>>> together roles into role catagories that you can assign to a specific >>>> user or user group. >>>> >>>> User Groups are different as you want to assign a set of permissions to >>>> a group of users. >>>> >>>> So composite roles are used to group together roles of a set of >>>> applications. User Groups are used to grant a set of perissions to a >>>> set of users. >>> Maybe it's just me, but I think of user groups as just a way to group >>> users, and roles as a way to group permissions. Roles are assigned to >>> user groups. Permissions are assigned to roles. >>> >> We dont' have the concept of a permission, so, assigning roles to a >> composite role is equivalent right now of assigning permissions to a role. > Isn't that what Pedro is working on right now? No. His is like: user in this group as write access to this document. This is just roles and sets of roles. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From ssilvert at redhat.com Wed Nov 4 12:24:39 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Wed, 04 Nov 2015 12:24:39 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <563A37AC.80604@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> <563A0B98.4080909@redhat.com> <563A12E6.2040206@redhat.com> <563A2395.8010900@redhat.com> <563A2657.1020209@redhat.com> <563A307D.4070302@redhat.com> <563A37AC.80604@redhat.com> Message-ID: <563A3F57.2080008@redhat.com> On 11/4/2015 11:51 AM, Bill Burke wrote: > > On 11/4/2015 11:21 AM, Stan Silvert wrote: >> On 11/4/2015 10:37 AM, Bill Burke wrote: >>> On 11/4/2015 10:26 AM, Stan Silvert wrote: >>>> On 11/4/2015 9:15 AM, Bill Burke wrote: >>>>> I've alread stated the reason for composite roles: >>>>> >>>>> Say you have a set of applications under the Sales and Marketing >>>>> Department: A Leads Application, Eloqua, and Salesforce. Each of the >>>>> applications has a set of roles that are used to manage access to >>>>> various features of each application. For example, each app might have >>>>> an "admin" role. You would then want to organize permissions into >>>>> categories and assign coarser grain roles to individual users. So, you >>>>> would create a "Sales Admin" composite role that contains the "admin" >>>>> role of each sales application. Composite roles allow you to group >>>>> together roles into role catagories that you can assign to a specific >>>>> user or user group. >>>>> >>>>> User Groups are different as you want to assign a set of permissions to >>>>> a group of users. >>>>> >>>>> So composite roles are used to group together roles of a set of >>>>> applications. User Groups are used to grant a set of perissions to a >>>>> set of users. >>>> Maybe it's just me, but I think of user groups as just a way to group >>>> users, and roles as a way to group permissions. Roles are assigned to >>>> user groups. Permissions are assigned to roles. >>>> >>> We dont' have the concept of a permission, so, assigning roles to a >>> composite role is equivalent right now of assigning permissions to a role. >> Isn't that what Pedro is working on right now? > No. His is like: user in this group as write access to this document. > This is just roles and sets of roles. > > "write access to this document" is a permission. Permissions assigned to roles. Roles assigned to groups. Maybe it's just semantics, but that's the kind of think I'm used to seeing in other systems. From sthorger at redhat.com Thu Nov 5 03:24:21 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 09:24:21 +0100 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <563A3F57.2080008@redhat.com> References: <5639216B.9050500@redhat.com> <228762845.2961154.1446588803670.JavaMail.zimbra@redhat.com> <1905736409.3325538.1446641201936.JavaMail.zimbra@redhat.com> <563A0B98.4080909@redhat.com> <563A12E6.2040206@redhat.com> <563A2395.8010900@redhat.com> <563A2657.1020209@redhat.com> <563A307D.4070302@redhat.com> <563A37AC.80604@redhat.com> <563A3F57.2080008@redhat.com> Message-ID: After our chat I realized I was confusing things. I think the initial design proposed by Bill is good, but we should keep composite roles. Roles should rename roles and we should introduce role namespaces to replace the current realm/client role split. So we should have: * Role * Composite role - these are powerful as they permit multiple inheritance (and we also deal with cyclic inheritance). I believe these works well and we've not had any negative feedback on them AFAIK * Role namespaces - ability to create a hierarchy of roles (a role namespace should just be a namespace, and not some sort of all inclusive role) * Group - Simple hierarchy (no multi inheritance) - A group has one or more attributes - A group has one or more role mappings - A user belongs to one or more groups - A user inherits all role mappings of the group - A user inherits all attributes of the group (we need to define how duplicates are resolved) I think this will provide the required tools to model most things. Obviously it doesn't model permissions, but that's not the agenda. Permissions are for Pedro's work, which will hopefully be brought into master sometime next year. With regards to how this is all added to tokens and managed in adapters we need to be able to: * Include/exclude groups in the token * Include/exclude roles in the token - Including ability to specify namespaces to include * For JEE adapters it should be possible to configure how roles from the token are mapped onto roles in the app - Full namespace - Include single namespace and only include role name, not full namespace - More (groups -> roles?)? Custom? Another related topic is fine-grained permissions in the admin console/endpoints. We should leave it as is for now (course grained), with the exception of utilizing role namespaces (instead of having to have "mock" clients). In the future we should be able to leverage Pedro's work to provide proper fine-grained permissions to the admin console/endpoints. On 4 November 2015 at 18:24, Stan Silvert wrote: > On 11/4/2015 11:51 AM, Bill Burke wrote: > > > > On 11/4/2015 11:21 AM, Stan Silvert wrote: > >> On 11/4/2015 10:37 AM, Bill Burke wrote: > >>> On 11/4/2015 10:26 AM, Stan Silvert wrote: > >>>> On 11/4/2015 9:15 AM, Bill Burke wrote: > >>>>> I've alread stated the reason for composite roles: > >>>>> > >>>>> Say you have a set of applications under the Sales and Marketing > >>>>> Department: A Leads Application, Eloqua, and Salesforce. Each of > the > >>>>> applications has a set of roles that are used to manage access to > >>>>> various features of each application. For example, each app might > have > >>>>> an "admin" role. You would then want to organize permissions into > >>>>> categories and assign coarser grain roles to individual users. So, > you > >>>>> would create a "Sales Admin" composite role that contains the "admin" > >>>>> role of each sales application. Composite roles allow you to group > >>>>> together roles into role catagories that you can assign to a specific > >>>>> user or user group. > >>>>> > >>>>> User Groups are different as you want to assign a set of permissions > to > >>>>> a group of users. > >>>>> > >>>>> So composite roles are used to group together roles of a set of > >>>>> applications. User Groups are used to grant a set of perissions to a > >>>>> set of users. > >>>> Maybe it's just me, but I think of user groups as just a way to group > >>>> users, and roles as a way to group permissions. Roles are assigned to > >>>> user groups. Permissions are assigned to roles. > >>>> > >>> We dont' have the concept of a permission, so, assigning roles to a > >>> composite role is equivalent right now of assigning permissions to a > role. > >> Isn't that what Pedro is working on right now? > > No. His is like: user in this group as write access to this document. > > This is just roles and sets of roles. > > > > > "write access to this document" is a permission. Permissions assigned > to roles. Roles assigned to groups. > > Maybe it's just semantics, but that's the kind of think I'm used to > seeing in other systems. > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/52ab3cab/attachment.html From sthorger at redhat.com Thu Nov 5 04:52:13 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 10:52:13 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes In-Reply-To: <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> Message-ID: The way to extend Keycloak is by implementing your own custom providers of the many SPIs we provide. Some SPIs are more stable (so marked as public) and others are not (so marked as private). If there are things that you want to customize that can't be done with an existing SPI then let us know and we may consider adding additional SPIs. On 4 November 2015 at 17:16, Erik Mulder wrote: > Thanks for your response! > > Indeed we already did a proof of concept where we added a custom mapper > the way you described (didn't know it was 'protected' territory :). The > question is: do we have to override the file > 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper > in the original project or is there another way where we don't need to > touch the original sources and keep all our changes in a separate > project? And how can we do it such that it stays easy to upgrade to > newer KeyCloak releases? > Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at the docs ( http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) and examples for other provider ( https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory ) > > As for JPA: it would be easier to integrate with the existing JPA > project. Again we are wondering whether to start modifying original > sources (like persistence.xml) or try to 'externalize' our changes > somehow and integrate them using existing 'hooks' in the system or maybe > merge projects during build. > > Maybe there is no good answer to this and we'll always be having some > manual merge pains when upgrading to new KeyCloak versions. We just > wanted to check if there are preferred ways to add functionality with > the least amount of impact on the original sources. > I initially wanted the ability to add custom entities to the JpaConnectProvider, but couldn't find a way to define entities programatically with JPA. To add your own persistence.xml you would have to define your own implementation of JpaConnectionProvider and change what is loaded by default (connectionsJpa/provider attribute in keycloak-server.json). Alternative, which is cleaner, but you end up with separate connection/transaction, is to create your own EntityManagerFactory. If it's only used by one provider (for example a custom UserFederationProvider) there's no need to add a connect provider (that's just a way to share one EntityManagerFactory between multiple providers) and you can just create it in the MyUserFederationProviderFactory. > > > On 04/11/15 15:30, Bill Burke wrote: > > Custom mappers should be possible. I didn't document it as I wasn't > > sure if we wanted to make the SPI public. Custom mappers should just > > follow the Provider SPI and they will be picked up. If you see the > > META-INF/services/... file in the resources directory of the "services" > > or "broker" modules you'll see how to set this up. > > > > As for extending the JPA datamodel, what you could do is write a new JPA > > Connections Provider and plug that in. See connections/jpa. I'm not > > sure how you would handle the liquibase db migration. > > > > On 11/4/2015 6:03 AM, Erik Mulder wrote: > >> Hi everybody, > >> > >> Quick intro: I?m part of a development team in The Netherlands that is > >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize > >> this and will use OpenID Connect to secure our REST services. It?s a > >> great product and seems to be the only one having both support for all > >> kinds of security standards and a model and GUI for users and roles. > >> Thanks for creating it! J > >> > >> (if this should be asked instead on the users mailing list, please > >> correct me and I?ll post it there) > >> > >> So far, so good, but we have some extra requirements that do not fit > >> into the base KeyCloak data model. See below for details if you?re > >> interested. My question is: what is the preferred way / best practice to > >> extend the functionality of KeyCloak while keeping the impact on the > >> original sources to a minimum? Of course we could just fork the most > >> recent version and start hacking away, but we?d like to be able to > >> upgrade to newer versions of KeyCloak without too much hassle. > >> Possibilities that we?ve come up with so far: > >> > >> 1.Create completely separate modules that will extend the functionality > >> the way we need. > >> > >> 2.Fork on Github, apply custom changes, and try to merge in updates from > >> the master / release branches / tags > >> > >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such > >> as Truezip > >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - > >> manipulate zip files by adding/removing/replacing or Shade > >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine > multiple > >> jars to 1 'uber-jar' containing the contents of both and when > >> overlapping decide on conflicts through configuration. > >> > >> Of course number 1 is preferred, but I do not see how to add custom > >> mappers or JPA entities without making changes in the original module > >> files. The other options seem like valid alternatives, but maybe there > >> is better / standard way to do this. So any help / insight / shared > >> experience on this is much appreciated, thanks! > >> > >> Kind regards, > >> > >> Erik Mulder > >> > >> Senior Software Engineer > >> > >> Docdata Payments ? NL > >> > >> P.S. Details on why we want to extend the KeyCloak data model: (any > >> feedback on the contents of this P.S. is also welcome!) > >> > >> Our clients are merchants that have several webshops. We manage their > >> online payments (shopping cart checkout). We want to be able to let a > >> merchant manage their own users and let a user have different roles for > >> different webshops within the same merchant. The overall possible roles > >> are fixed though, no specific roles per merchant. We could create a > >> separate realm for every merchant, but then we need to duplicate all > >> roles every time. Furthermore, in KeyCloak there is no concept of a role > >> within a certain context. This is very understandable, since every > >> situation has it?s own requirements. We did a proof of concept by adding > >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. > >> and adding a custom mapper that can put this information on the Access > >> token. Worked like a charm! But it does need some changes in the > >> KeyCloak modules and sources to work, hence the question above. > >> > >> > >> > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/15760ecc/attachment-0001.html From psilva at redhat.com Thu Nov 5 05:06:24 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Thu, 5 Nov 2015 05:06:24 -0500 (EST) Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes In-Reply-To: References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> Message-ID: <469783768.4549663.1446717984477.JavaMail.zimbra@redhat.com> Btw, I've already worked on a SPI to plug custom endpoints to both Admin and Realm API. I'm still holding a PR with these changes, but if you need it I can send them. Just to avoid re-work :) ----- Original Message ----- From: "Stian Thorgersen" To: "Erik Mulder" Cc: keycloak-dev at lists.jboss.org Sent: Thursday, November 5, 2015 7:52:13 AM Subject: Re: [keycloak-dev] Preferred way to make KeyCloak custom changes The way to extend Keycloak is by implementing your own custom providers of the many SPIs we provide. Some SPIs are more stable (so marked as public) and others are not (so marked as private). If there are things that you want to customize that can't be done with an existing SPI then let us know and we may consider adding additional SPIs. On 4 November 2015 at 17:16, Erik Mulder < erik.mulder at docdatapayments.com > wrote: Thanks for your response! Indeed we already did a proof of concept where we added a custom mapper the way you described (didn't know it was 'protected' territory :). The question is: do we have to override the file 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper in the original project or is there another way where we don't need to touch the original sources and keep all our changes in a separate project? And how can we do it such that it stays easy to upgrade to newer KeyCloak releases? Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at the docs ( http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html ) and examples for other provider ( https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory ) As for JPA: it would be easier to integrate with the existing JPA project. Again we are wondering whether to start modifying original sources (like persistence.xml) or try to 'externalize' our changes somehow and integrate them using existing 'hooks' in the system or maybe merge projects during build. Maybe there is no good answer to this and we'll always be having some manual merge pains when upgrading to new KeyCloak versions. We just wanted to check if there are preferred ways to add functionality with the least amount of impact on the original sources. I initially wanted the ability to add custom entities to the JpaConnectProvider, but couldn't find a way to define entities programatically with JPA. To add your own persistence.xml you would have to define your own implementation of JpaConnectionProvider and change what is loaded by default (connectionsJpa/provider attribute in keycloak-server.json). Alternative, which is cleaner, but you end up with separate connection/transaction, is to create your own EntityManagerFactory. If it's only used by one provider (for example a custom UserFederationProvider) there's no need to add a connect provider (that's just a way to share one EntityManagerFactory between multiple providers) and you can just create it in the MyUserFederationProviderFactory. On 04/11/15 15:30, Bill Burke wrote: > Custom mappers should be possible. I didn't document it as I wasn't > sure if we wanted to make the SPI public. Custom mappers should just > follow the Provider SPI and they will be picked up. If you see the > META-INF/services/... file in the resources directory of the "services" > or "broker" modules you'll see how to set this up. > > As for extending the JPA datamodel, what you could do is write a new JPA > Connections Provider and plug that in. See connections/jpa. I'm not > sure how you would handle the liquibase db migration. > > On 11/4/2015 6:03 AM, Erik Mulder wrote: >> Hi everybody, >> >> Quick intro: I?m part of a development team in The Netherlands that is >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >> this and will use OpenID Connect to secure our REST services. It?s a >> great product and seems to be the only one having both support for all >> kinds of security standards and a model and GUI for users and roles. >> Thanks for creating it! J >> >> (if this should be asked instead on the users mailing list, please >> correct me and I?ll post it there) >> >> So far, so good, but we have some extra requirements that do not fit >> into the base KeyCloak data model. See below for details if you?re >> interested. My question is: what is the preferred way / best practice to >> extend the functionality of KeyCloak while keeping the impact on the >> original sources to a minimum? Of course we could just fork the most >> recent version and start hacking away, but we?d like to be able to >> upgrade to newer versions of KeyCloak without too much hassle. >> Possibilities that we?ve come up with so far: >> >> 1.Create completely separate modules that will extend the functionality >> the way we need. >> >> 2.Fork on Github, apply custom changes, and try to merge in updates from >> the master / release branches / tags >> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >> as Truezip >> ( http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html ) - >> manipulate zip files by adding/removing/replacing or Shade >> ( http://maven.apache.org/plugins/maven-shade-plugin/ ) - combine multiple >> jars to 1 'uber-jar' containing the contents of both and when >> overlapping decide on conflicts through configuration. >> >> Of course number 1 is preferred, but I do not see how to add custom >> mappers or JPA entities without making changes in the original module >> files. The other options seem like valid alternatives, but maybe there >> is better / standard way to do this. So any help / insight / shared >> experience on this is much appreciated, thanks! >> >> Kind regards, >> >> Erik Mulder >> >> Senior Software Engineer >> >> Docdata Payments ? NL >> >> P.S. Details on why we want to extend the KeyCloak data model: (any >> feedback on the contents of this P.S. is also welcome!) >> >> Our clients are merchants that have several webshops. We manage their >> online payments (shopping cart checkout). We want to be able to let a >> merchant manage their own users and let a user have different roles for >> different webshops within the same merchant. The overall possible roles >> are fixed though, no specific roles per merchant. We could create a >> separate realm for every merchant, but then we need to duplicate all >> roles every time. Furthermore, in KeyCloak there is no concept of a role >> within a certain context. This is very understandable, since every >> situation has it?s own requirements. We did a proof of concept by adding >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >> and adding a custom mapper that can put this information on the Access >> token. Worked like a charm! But it does need some changes in the >> KeyCloak modules and sources to work, hence the question above. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From sthorger at redhat.com Thu Nov 5 05:33:15 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 11:33:15 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <5639253A.7090408@redhat.com> References: <5639253A.7090408@redhat.com> Message-ID: I don't think we should use the group concept for managing permissions in admin endpoints. That'll just be to confusing IMO. In either case we don't need a distinction between user and client, as we'd have roles for that. What about domain/organization or something? Within a realm you could have one or more organizations. Users and clients belong to a single organization. Then we'd have a role namespace for the organiztion for example roles would be: * org.keycloak//view-clients * org.keycloak//manage-clients * org.keycloak//view-users * org.keycloak//manage-users In fact you don't need groups to do that, just role namespaces and the ability to configure what "namespace" is used for a particular client or user. On 3 November 2015 at 22:20, Bill Burke wrote: > In my previous email I talked about combining Groups and Role > Namespaces. Now I want to talk about User Groups vs. Client Groups. > > User Groups would manage a set of users. Members would automatically > inherit a set of "permissions": a set of roles. User Groups would also > provide a set of attributes that the user inherits. > > I'd like to introduce the concept of a Client Group. Client Group would > have: > > * Roles - basically a role namespace > * Permissions - set of roles service accounts members inherit > * Scope - same as our current concept of scope > * Protocol Policies - common protocol configuration. > > Each Client Group would have some default roles defined. i.e. roles > that allow a user to edit any client in the client group. > > Each Client would have the same configuration options. They would be > able to have an additional set of roles, permissions, scope, and > overridable Protocol Policies. > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/4616ba70/attachment.html From sthorger at redhat.com Thu Nov 5 05:34:00 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 11:34:00 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: References: <5639253A.7090408@redhat.com> Message-ID: However, maybe we should not try to introduce more fine-grained permissions in admin endpoints at the moment. In the future we should be able to leverage Pedro's work and do it properly. On 5 November 2015 at 11:33, Stian Thorgersen wrote: > I don't think we should use the group concept for managing permissions in > admin endpoints. That'll just be to confusing IMO. In either case we don't > need a distinction between user and client, as we'd have roles for that. > > What about domain/organization or something? Within a realm you could have > one or more organizations. Users and clients belong to a single > organization. Then we'd have a role namespace for the organiztion for > example roles would be: > > * org.keycloak//view-clients > * org.keycloak//manage-clients > * org.keycloak//view-users > * org.keycloak//manage-users > > In fact you don't need groups to do that, just role namespaces and the > ability to configure what "namespace" is used for a particular client or > user. > > On 3 November 2015 at 22:20, Bill Burke wrote: > >> In my previous email I talked about combining Groups and Role >> Namespaces. Now I want to talk about User Groups vs. Client Groups. >> >> User Groups would manage a set of users. Members would automatically >> inherit a set of "permissions": a set of roles. User Groups would also >> provide a set of attributes that the user inherits. >> >> I'd like to introduce the concept of a Client Group. Client Group would >> have: >> >> * Roles - basically a role namespace >> * Permissions - set of roles service accounts members inherit >> * Scope - same as our current concept of scope >> * Protocol Policies - common protocol configuration. >> >> Each Client Group would have some default roles defined. i.e. roles >> that allow a user to edit any client in the client group. >> >> Each Client would have the same configuration options. They would be >> able to have an additional set of roles, permissions, scope, and >> overridable Protocol Policies. >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/612f4fca/attachment.html From mposolda at redhat.com Thu Nov 5 05:46:09 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 5 Nov 2015 11:46:09 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management Message-ID: <563B3371.8040402@redhat.com> I wonder if we should improve handling of custom attributes in freemarker templates and their validation. Let's assume that I want to add custom attribute "birthday" and I want to add it to all screens where user can create/edit account. I can see those issues: - Admin will need to edit 3 separate freemarker templates (registration, account management, update profile page) to have this attribute displayed on all those places. - Missing validations. For registration screen, admin can implement his own validator through the Authentication SPI. But for account management and update profile it's currently no way to add custom validations. But for my "birthday" field, I usually want to have same custom validation applied on all 3 places - Built-in validations: Similarly like we have builtin password policies, we can provide some builtin validators for custom fields (mandatory, regex) I wonder if we should add the tab in admin console where people can specify: - Name of custom attribute and the type. Then we have some freemarker helper, which will render those attributes in all 3 places (registration, account mgmt, updateProfile). I believe something similar like Bill did for custom providers in admin console ( kc-provider-config directive) can work here too. - Define the validators applied to each field. Similarly like we have password/OTP policies, we can have "policies" for each custom attribute. We can provide "mandatory" and "regex" policy with possibility for people to add custom field validator implementations if they want. Not sure if this was already discussed before and just not implemented. However I think this will be useful and currently can't think of much reasons why it couldn't work? Marek From sthorger at redhat.com Thu Nov 5 06:03:52 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 12:03:52 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: <563B3371.8040402@redhat.com> References: <563B3371.8040402@redhat.com> Message-ID: +1 I'd like to see the ability to define a "user profile" for the realm with associated validators. This would also allow having a update profile required action that could display a form if there are missing or invalid profile information. On 5 November 2015 at 11:46, Marek Posolda wrote: > I wonder if we should improve handling of custom attributes in > freemarker templates and their validation. Let's assume that I want to > add custom attribute "birthday" and I want to add it to all screens > where user can create/edit account. I can see those issues: > > - Admin will need to edit 3 separate freemarker templates (registration, > account management, update profile page) to have this attribute > displayed on all those places. > > - Missing validations. For registration screen, admin can implement his > own validator through the Authentication SPI. But for account management > and update profile it's currently no way to add custom validations. But > for my "birthday" field, I usually want to have same custom validation > applied on all 3 places > > - Built-in validations: Similarly like we have builtin password > policies, we can provide some builtin validators for custom fields > (mandatory, regex) > > > I wonder if we should add the tab in admin console where people can > specify: > > - Name of custom attribute and the type. Then we have some freemarker > helper, which will render those attributes in all 3 places > (registration, account mgmt, updateProfile). I believe something similar > like Bill did for custom providers in admin console ( kc-provider-config > directive) can work here too. > > - Define the validators applied to each field. Similarly like we have > password/OTP policies, we can have "policies" for each custom attribute. > We can provide "mandatory" and "regex" policy with possibility for > people to add custom field validator implementations if they want. > > > Not sure if this was already discussed before and just not implemented. > However I think this will be useful and currently can't think of much > reasons why it couldn't work? > > Marek > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/23dc111e/attachment.html From mposolda at redhat.com Thu Nov 5 06:12:44 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 5 Nov 2015 12:12:44 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: References: <5639253A.7090408@redhat.com> Message-ID: <563B39AC.3090708@redhat.com> I like also the possibility to have "common" set of protocol mappers shared between more clients. Also in my client application, I may want to share roles defined in 2 different namespaces. So IMO it will be good if client can be added to more different client groups/organizations and inherit common roles, protocol mappers and scopes from all of them. Marek On 05/11/15 11:34, Stian Thorgersen wrote: > However, maybe we should not try to introduce more fine-grained > permissions in admin endpoints at the moment. In the future we should > be able to leverage Pedro's work and do it properly. > > On 5 November 2015 at 11:33, Stian Thorgersen > wrote: > > I don't think we should use the group concept for managing > permissions in admin endpoints. That'll just be to confusing IMO. > In either case we don't need a distinction between user and > client, as we'd have roles for that. > > What about domain/organization or something? Within a realm you > could have one or more organizations. Users and clients belong to > a single organization. Then we'd have a role namespace for the > organiztion for example roles would be: > > * org.keycloak//view-clients > * org.keycloak//manage-clients > * org.keycloak//view-users > * org.keycloak//manage-users > > In fact you don't need groups to do that, just role namespaces and > the ability to configure what "namespace" is used for a particular > client or user. > > On 3 November 2015 at 22:20, Bill Burke > wrote: > > In my previous email I talked about combining Groups and Role > Namespaces. Now I want to talk about User Groups vs. Client > Groups. > > User Groups would manage a set of users. Members would > automatically > inherit a set of "permissions": a set of roles. User Groups > would also > provide a set of attributes that the user inherits. > > I'd like to introduce the concept of a Client Group. Client > Group would > have: > > * Roles - basically a role namespace > * Permissions - set of roles service accounts members inherit > * Scope - same as our current concept of scope > * Protocol Policies - common protocol configuration. > > Each Client Group would have some default roles defined. i.e. > roles > that allow a user to edit any client in the client group. > > Each Client would have the same configuration options. They > would be > able to have an additional set of roles, permissions, scope, and > overridable Protocol Policies. > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/8ef7297c/attachment.html From sthorger at redhat.com Thu Nov 5 06:16:56 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 12:16:56 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <563B39AC.3090708@redhat.com> References: <5639253A.7090408@redhat.com> <563B39AC.3090708@redhat.com> Message-ID: Ignore my previous response - I completely miss-read what was being proposed ;) On 5 November 2015 at 12:12, Marek Posolda wrote: > I like also the possibility to have "common" set of protocol mappers > shared between more clients. Also in my client application, I may want to > share roles defined in 2 different namespaces. So IMO it will be good if > client can be added to more different client groups/organizations and > inherit common roles, protocol mappers and scopes from all of them. > > Marek > > > On 05/11/15 11:34, Stian Thorgersen wrote: > > However, maybe we should not try to introduce more fine-grained > permissions in admin endpoints at the moment. In the future we should be > able to leverage Pedro's work and do it properly. > > On 5 November 2015 at 11:33, Stian Thorgersen wrote: > >> I don't think we should use the group concept for managing permissions in >> admin endpoints. That'll just be to confusing IMO. In either case we don't >> need a distinction between user and client, as we'd have roles for that. >> >> What about domain/organization or something? Within a realm you could >> have one or more organizations. Users and clients belong to a single >> organization. Then we'd have a role namespace for the organiztion for >> example roles would be: >> >> * org.keycloak//view-clients >> * org.keycloak//manage-clients >> * org.keycloak//view-users >> * org.keycloak//manage-users >> >> In fact you don't need groups to do that, just role namespaces and the >> ability to configure what "namespace" is used for a particular client or >> user. >> >> On 3 November 2015 at 22:20, Bill Burke < >> bburke at redhat.com> wrote: >> >>> In my previous email I talked about combining Groups and Role >>> Namespaces. Now I want to talk about User Groups vs. Client Groups. >>> >>> User Groups would manage a set of users. Members would automatically >>> inherit a set of "permissions": a set of roles. User Groups would also >>> provide a set of attributes that the user inherits. >>> >>> I'd like to introduce the concept of a Client Group. Client Group would >>> have: >>> >>> * Roles - basically a role namespace >>> * Permissions - set of roles service accounts members inherit >>> * Scope - same as our current concept of scope >>> * Protocol Policies - common protocol configuration. >>> >>> Each Client Group would have some default roles defined. i.e. roles >>> that allow a user to edit any client in the client group. >>> >>> Each Client would have the same configuration options. They would be >>> able to have an additional set of roles, permissions, scope, and >>> overridable Protocol Policies. >>> >>> -- >>> Bill Burke >>> JBoss, a division of Red Hat >>> http://bill.burkecentral.com >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/119cea20/attachment-0001.html From sthorger at redhat.com Thu Nov 5 06:23:15 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 12:23:15 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <5639253A.7090408@redhat.com> References: <5639253A.7090408@redhat.com> Message-ID: On 3 November 2015 at 22:20, Bill Burke wrote: > In my previous email I talked about combining Groups and Role > Namespaces. Now I want to talk about User Groups vs. Client Groups. > > User Groups would manage a set of users. Members would automatically > inherit a set of "permissions": a set of roles. User Groups would also > provide a set of attributes that the user inherits. > Permission != role > > I'd like to introduce the concept of a Client Group. Client Group would > have: > > * Roles - basically a role namespace > -1 Having roles tied to a client or client group is exactly what we should go away from. IMO role namespaces should be a completely separate thing. > * Permissions - set of roles service accounts members inherit > -1 It shouldn't be called permissions, it would be role mappings. In either case a service account is backed by a regular user, which can be part of a user group and would get role mappings from there. > * Scope - same as our current concept of scope * Protocol Policies - common protocol configuration. > +1 To scope and protocol policies > > Each Client Group would have some default roles defined. i.e. roles > that allow a user to edit any client in the client group. > I don't understand this > > Each Client would have the same configuration options. They would be > able to have an additional set of roles, permissions, scope, and > overridable Protocol Policies. Same comment as above - why would a client have roles/permissions? I assume we where moving away from that with role namespaces > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/b9bbc663/attachment.html From paul.wolf23 at googlemail.com Thu Nov 5 08:43:11 2015 From: paul.wolf23 at googlemail.com (Paul Wolf) Date: Thu, 5 Nov 2015 14:43:11 +0100 Subject: [keycloak-dev] Password grant flow with client consent is bugged Message-ID: Hi guys, I am currently evaluating Keycloak and so far I have only two complaints: 1. The implicit flow is not supported, but I already found a Jira issue for that. How realistic is it that the feature comes with 1.7CR1? 2. When the client consent is needed for an application, which uses the password grant flow, the flow fails all the time... Even if the consent has been given. This seems an easy to fix bug. Will you fix that? Should I write a fix and make a pull request? Otherwise good stuff so far :) Best regards, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/6db61ae7/attachment.html From bburke at redhat.com Thu Nov 5 08:43:25 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 5 Nov 2015 08:43:25 -0500 Subject: [keycloak-dev] roles vs. groups In-Reply-To: <412784568.3422856.1446647787493.JavaMail.zimbra@redhat.com> References: <5639216B.9050500@redhat.com> <1447226686.2954301.1446588117003.JavaMail.zimbra@redhat.com> <56395BA3.6020509@redhat.com> <539688376.3287936.1446636768515.JavaMail.zimbra@redhat.com> <563A102C.8070701@redhat.com> <412784568.3422856.1446647787493.JavaMail.zimbra@redhat.com> Message-ID: <563B5CFD.70306@redhat.com> On 11/4/2015 9:36 AM, Pedro Igor Silva wrote: >> There really is no equivalent in Java EE for leveraging a >> User/Role/Group relationship. We'd have to add it to our adapters. >> Isn't "users of Group A have the manager role" enough? > > For most cases yes. But if you want to be more flexible you can support that as well. IIRC, Hawkular wants group role. And it might be useful for others as well. > Doesn't User/Role/Group start to overlap with what you're doing? I thought Hawkular wanted a permission model so that you could assign permissions to resources based on group membership. If I added a user/role/group relationship mapping, wouldn't people start using that to implement similar permission model to what you are doing? -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Thu Nov 5 09:01:03 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 5 Nov 2015 09:01:03 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: References: <5639253A.7090408@redhat.com> Message-ID: <563B611F.2070304@redhat.com> On 11/5/2015 6:23 AM, Stian Thorgersen wrote: > > > On 3 November 2015 at 22:20, Bill Burke > wrote: > > In my previous email I talked about combining Groups and Role > Namespaces. Now I want to talk about User Groups vs. Client Groups. > > User Groups would manage a set of users. Members would automatically > inherit a set of "permissions": a set of roles. User Groups would also > provide a set of attributes that the user inherits. > > > Permission != role > > > I'd like to introduce the concept of a Client Group. Client Group would > have: > > * Roles - basically a role namespace > > > -1 Having roles tied to a client or client group is exactly what we > should go away from. IMO role namespaces should be a completely separate > thing. > I don't agree at all. If User Groups and Client Groups exist, there is no need for role namespaces. It is stupid to have to create another concept (role namespace) to define the roles one specific client or a group of clients expects. If you combine Role namespace and Groups you can define things like a group admin role. Roles that mean something to the group. > Each Client Group would have some default roles defined. i.e. roles > that allow a user to edit any client in the client group. > > > I don't understand this > A Client Group could have a "client group admin" role. If a user has that role it can manage clients in the group. Another role might be "client membership admin". This role allows a user to add or remove clients from the group. Conversely, user groups could have a "user group admin". When granted, this role allows a user to manage users in the group. YOu can also do things like define a "Manager" role for the group. This "Manager" would be granted "user group admin" privileges and also granted access to other systems like "HR", "Attendence", "Benefits", etc. I think this permission concept should be built into Keycloak as it is a core feature. I'll write u a separate email about this. > > Each Client would have the same configuration options. They would be > able to have an additional set of roles, permissions, scope, and > overridable Protocol Policies. > > > Same comment as above - why would a client have roles/permissions? I > assume we where moving away from that with role namespaces > Again, I think role namespaces are redundant. A client can define a set of roles that it offers. A service account (the client) can have roles associated with it so it can do certain actions. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From thomas.raehalme at aitiofinland.com Thu Nov 5 09:01:41 2015 From: thomas.raehalme at aitiofinland.com (Thomas Raehalme) Date: Thu, 5 Nov 2015 16:01:41 +0200 Subject: [keycloak-dev] adapter-core and errors Message-ID: Hi, I'm playing around with client-side error pages regarding Keycloak authentication. I have an Java EE web application utilizing error-page definitions in web.xml. If authentication fails for a reason or another, shouldn't the adapter-core enable the use of these error-pages? At the moment all or most of the AuthChallenge implementations in adapter-core call the setStatus method of the HttpFacade. This results in blank responses with only the status being set. If sendError was used instead then the default error-page handling would kick in. Is there a reason I am missing why using setStatus instead of sendError in, for example, OAuthRequestAuthenticator? Best regards, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/51d99632/attachment.html From velias at redhat.com Thu Nov 5 09:12:46 2015 From: velias at redhat.com (Vlastimil Elias) Date: Thu, 5 Nov 2015 15:12:46 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: <563B3371.8040402@redhat.com> References: <563B3371.8040402@redhat.com> Message-ID: <563B63DE.9060700@redhat.com> +1, server side validations of user profile attributes and their use on all necessary places is a must. Freemarker helper to be used in templates for basic presentation of fields is welcomed also, user can create his own template still if he needs some nicer formatting of the forms. Cheers Vlastimil On 5.11.2015 11:46, Marek Posolda wrote: > I wonder if we should improve handling of custom attributes in > freemarker templates and their validation. Let's assume that I want to > add custom attribute "birthday" and I want to add it to all screens > where user can create/edit account. I can see those issues: > > - Admin will need to edit 3 separate freemarker templates (registration, > account management, update profile page) to have this attribute > displayed on all those places. > > - Missing validations. For registration screen, admin can implement his > own validator through the Authentication SPI. But for account management > and update profile it's currently no way to add custom validations. But > for my "birthday" field, I usually want to have same custom validation > applied on all 3 places > > - Built-in validations: Similarly like we have builtin password > policies, we can provide some builtin validators for custom fields > (mandatory, regex) > > > I wonder if we should add the tab in admin console where people can specify: > > - Name of custom attribute and the type. Then we have some freemarker > helper, which will render those attributes in all 3 places > (registration, account mgmt, updateProfile). I believe something similar > like Bill did for custom providers in admin console ( kc-provider-config > directive) can work here too. > > - Define the validators applied to each field. Similarly like we have > password/OTP policies, we can have "policies" for each custom attribute. > We can provide "mandatory" and "regex" policy with possibility for > people to add custom field validator implementations if they want. > > > Not sure if this was already discussed before and just not implemented. > However I think this will be useful and currently can't think of much > reasons why it couldn't work? > > Marek > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -- Vlastimil Elias Principal Software Engineer Developer Portal Engineering Team From psilva at redhat.com Thu Nov 5 09:26:28 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Thu, 5 Nov 2015 09:26:28 -0500 (EST) Subject: [keycloak-dev] roles vs. groups In-Reply-To: <563B5CFD.70306@redhat.com> References: <5639216B.9050500@redhat.com> <1447226686.2954301.1446588117003.JavaMail.zimbra@redhat.com> <56395BA3.6020509@redhat.com> <539688376.3287936.1446636768515.JavaMail.zimbra@redhat.com> <563A102C.8070701@redhat.com> <412784568.3422856.1446647787493.JavaMail.zimbra@redhat.com> <563B5CFD.70306@redhat.com> Message-ID: <1582744239.4728822.1446733588420.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Bill Burke" > To: "Pedro Igor Silva" > Cc: keycloak-dev at lists.jboss.org > Sent: Thursday, November 5, 2015 11:43:25 AM > Subject: Re: [keycloak-dev] roles vs. groups > > > > On 11/4/2015 9:36 AM, Pedro Igor Silva wrote: > >> There really is no equivalent in Java EE for leveraging a > >> User/Role/Group relationship. We'd have to add it to our adapters. > >> Isn't "users of Group A have the manager role" enough? > > > > For most cases yes. But if you want to be more flexible you can support > > that as well. IIRC, Hawkular wants group role. And it might be useful for > > others as well. > > > > Doesn't User/Role/Group start to overlap with what you're doing? I > thought Hawkular wanted a permission model so that you could assign > permissions to resources based on group membership. If I added a > user/role/group relationship mapping, wouldn't people start using that > to implement similar permission model to what you are doing? Not really. What I'm doing is completely different and is related with permissions and fine-grained authorization. One thing is say that User A has Role B. Or User A has Role B in Group C. Or User A is member of Group C. You are not assigning permissions, but defining a baseline to actually define them. Let's say you are logically separating permissions but not defining what, how and when can be accessed. Like I said in other emails, whatever you define for roles and groups would be used to create policies, covering fine-grained permissions and considering different types of access control mechanisms such as role-based, group-based, user-based, context-based, attribute-based, etc. Let's use the following example. You have defined that User A has Role B in Group C. From this statement you don't know what that means from a access control perspectice. You only know that User A has Role B in Group C. What that means from a access control mechanism is implicit and will probably be used to actually *enforce* a policy when accessing some resource. Usually, what people do is obtain that statement somewhere (eg.: from a token) and write a if statement around the code the will access a resource. Now, in the case of the Authz Server, you would not use ifs or enforce these policies by your self. You will use a rich and powerful policy engine to say if a resource can be accessed or not. Where you can use not only that statement (user/group/role) but mix other statements such as: user must have authenticated with OTP, time must be XX:XX:XX, the authenticated user must be the resource owner, only the Action X can be performed, etc. Or you can even just ask the authorization server for all the *entitlements* for an user. And the server will give you which resources and what actions can be accessed/performed by a given user. Regarding Hawkular, they need a permission model, yes. But they also need a permission model that can also support policies/permissions based on User/Role/Group relationship. > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > From bburke at redhat.com Thu Nov 5 10:36:23 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 5 Nov 2015 10:36:23 -0500 Subject: [keycloak-dev] adapter-core and errors In-Reply-To: References: Message-ID: <563B7777.8080906@redhat.com> Log a JIRA please. This is something I saw when doing SAML adapter and should make it uniform across all adapters. On 11/5/2015 9:01 AM, Thomas Raehalme wrote: > Hi, > > I'm playing around with client-side error pages regarding Keycloak > authentication. > > I have an Java EE web application utilizing error-page definitions in > web.xml. If authentication fails for a reason or another, shouldn't the > adapter-core enable the use of these error-pages? > > At the moment all or most of the AuthChallenge implementations in > adapter-core call the setStatus method of the HttpFacade. This results > in blank responses with only the status being set. If sendError was used > instead then the default error-page handling would kick in. > > Is there a reason I am missing why using setStatus instead of sendError > in, for example, OAuthRequestAuthenticator? > > Best regards, > Thomas > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From mposolda at redhat.com Thu Nov 5 10:50:06 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 5 Nov 2015 16:50:06 +0100 Subject: [keycloak-dev] Password grant flow with client consent is bugged In-Reply-To: References: Message-ID: <563B7AAE.4070403@redhat.com> On 05/11/15 14:43, Paul Wolf wrote: > > Hi guys, > > I am currently evaluating Keycloak and so far I have only two complaints: > 1. The implicit flow is not supported, but I already found a Jira > issue for that. How realistic is it that the feature comes with 1.7CR1? > It's planned and very realistic that it really happens :-) > > 2. When the client consent is needed for an application, which uses > the password grant flow, the flow fails all the time... Even if the > consent has been given. This seems an easy to fix bug. Will you fix > that? Should I write a fix and make a pull request? > +1 makes sense to me to let the password grant flow happen if consent has been already given. Feel free to create JIRA and PR will be even better. Current checking of consent is done in AuthenticationManager.actionRequired method. I hope you can extract the consent checking to separate method and reuse it. Marek > > Otherwise good stuff so far :) > > Best regards, > Paul > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/0177e699/attachment.html From bburke at redhat.com Thu Nov 5 11:11:43 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 5 Nov 2015 11:11:43 -0500 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: <563B3371.8040402@redhat.com> References: <563B3371.8040402@redhat.com> Message-ID: <563B7FBF.8060701@redhat.com> On 11/5/2015 5:46 AM, Marek Posolda wrote: > I wonder if we should improve handling of custom attributes in > freemarker templates and their validation. Let's assume that I want to > add custom attribute "birthday" and I want to add it to all screens > where user can create/edit account. I can see those issues: > > - Admin will need to edit 3 separate freemarker templates (registration, > account management, update profile page) to have this attribute > displayed on all those places. > we've discussed this before. The problem is formatting in each of the UIs. Often ordering of attributes is important. Grouping of attributes is a must too. i.e. Address, State, Country, Postal Code These 4 attribute must be grouped together, and address must come before state, and so on. You might also have a "Billing Address" group that needs to come after Home Address group. So, we'd have an automatic way for displaying attributes, then the developer would think "that looks like shit" and want to format things himself. Stian seems to think that CSS will solve this problem, but I'm not convinced. Also the look and feel could be quite different between registration, update profile, account management, and the admin profile. IMO, it would end up being easier to just edit the freemarker pages directly than to have to define each attribute and how it is grouped, ordered, and displayed on each of the pages within an admin console UI. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From thomas.raehalme at aitiofinland.com Thu Nov 5 11:34:43 2015 From: thomas.raehalme at aitiofinland.com (Thomas Raehalme) Date: Thu, 5 Nov 2015 18:34:43 +0200 Subject: [keycloak-dev] adapter-core and errors In-Reply-To: <563B7777.8080906@redhat.com> References: <563B7777.8080906@redhat.com> Message-ID: Ok, will do! I'll see if I can submit a PR as well. Best regards, Thomas On Nov 5, 2015 17:36, "Bill Burke" wrote: > Log a JIRA please. This is something I saw when doing SAML adapter and > should make it uniform across all adapters. > > On 11/5/2015 9:01 AM, Thomas Raehalme wrote: > > Hi, > > > > I'm playing around with client-side error pages regarding Keycloak > > authentication. > > > > I have an Java EE web application utilizing error-page definitions in > > web.xml. If authentication fails for a reason or another, shouldn't the > > adapter-core enable the use of these error-pages? > > > > At the moment all or most of the AuthChallenge implementations in > > adapter-core call the setStatus method of the HttpFacade. This results > > in blank responses with only the status being set. If sendError was used > > instead then the default error-page handling would kick in. > > > > Is there a reason I am missing why using setStatus instead of sendError > > in, for example, OAuthRequestAuthenticator? > > > > Best regards, > > Thomas > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/dc7ea096/attachment.html From bburke at redhat.com Thu Nov 5 12:31:00 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 5 Nov 2015 12:31:00 -0500 Subject: [keycloak-dev] user group admins Message-ID: <563B9254.1090209@redhat.com> A few users have been asking about the ability to limit the admin to managing a set of users in a group. I know Pedro is doing permission work, but IMO, this type of thing needs to be integrated and natural to the Keycloak UI as it would be a fundamental feature. Here's a proposal though based on my layout of User Groups in a previous email. Group Admins need to be able to: * Manage group membership * Manage users within a group to do things like reset credentials and other user management actions. * Grant roles to users within a group * Add attributes to the group * Grant roles to the group So, if each User Group had its own Role Namespace, we could define one or more roles that grant each of those permissions. i.e. "group-membership-admin", "group-user-admin", "group-admin". If a User Gruop has its own Role Namespace, it becomes really easy to compose adminstrative access. So you could define individual "admin groups" and grant users in those groups permission to manage one more groups. If groups don't have their own Role Namespace, you need a way to associate a role to each group admin permission. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Thu Nov 5 12:31:07 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 5 Nov 2015 12:31:07 -0500 Subject: [keycloak-dev] controlling which roles an admin can grant Message-ID: <563B925B.9030908@redhat.com> One of things that we need to be able to do if we have the idea of a "Group Admin" is to control specifically which role mappings an admin is allowed to grant. One of the places this comes up currently is that if an admin has the "manage-users" role, they can pretty much add any permission they want to themselves and get access to the whole realm. IMO, this is something we need now. It needs to be built into our admin UI. So, how could we add the ability to control which roles an admin is allowed to grant? Under the "Roles" menu option there would be a "Grant Permissions" tab. Here, the admin can select a role and specify a list of roles that can be granted if a user has that role. Here's an example: Let's say there are 2 sales applications "reporting" and "analytics". Each of the apps has defined an "admin" and "user" role. We want to have a developer manage user access to these systems. 1. Define "Sales Access Control Manager" role. 2. Go into "Roles" menu 3. Go to the "Role Granting Permissions" tab. 4. Select the "Sales Access Control Manager" role 5. Select and add the "reporting.user", "reporting.admin", "analytics.user", and "analytics.admin" roles to the list of roles a "Sales Access Control Manager" is allowed to grant. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Thu Nov 5 13:44:45 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 19:44:45 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <563B611F.2070304@redhat.com> References: <5639253A.7090408@redhat.com> <563B611F.2070304@redhat.com> Message-ID: On 5 November 2015 at 15:01, Bill Burke wrote: > > > On 11/5/2015 6:23 AM, Stian Thorgersen wrote: > >> >> >> On 3 November 2015 at 22:20, Bill Burke > > wrote: >> >> In my previous email I talked about combining Groups and Role >> Namespaces. Now I want to talk about User Groups vs. Client Groups. >> >> User Groups would manage a set of users. Members would automatically >> inherit a set of "permissions": a set of roles. User Groups would >> also >> provide a set of attributes that the user inherits. >> >> >> Permission != role >> >> >> I'd like to introduce the concept of a Client Group. Client Group >> would >> have: >> >> * Roles - basically a role namespace >> >> >> -1 Having roles tied to a client or client group is exactly what we >> should go away from. IMO role namespaces should be a completely separate >> thing. >> >> > I don't agree at all. If User Groups and Client Groups exist, there is no > need for role namespaces. It is stupid to have to create another concept > (role namespace) to define the roles one specific client or a group of > clients expects. > I've never the concept of realm and client roles. It's been difficult to explain and strange to use. I've always just used realm roles. It's a strange and limiting concept. Introducing client groups with further places to define roles just makes matters even worse. Now users have two go 3 different places to define roles: * Realm * Client Groups * Clients What happens if a client group and a client both have the same role by the way? It's a strange limitation. At least personally if I was using Keycloak I would simply use realm roles alone and define my own hierarchy with a delimiter. It's much better to have a single place to define roles, under the roles tab. Then allow users can define the namespaces/hierarchy they want. Role namespaces are easier to deal with and at the same time more flexible. I just don't see any reason why we would have roles specific to a client or client group. > > If you combine Role namespace and Groups you can define things like a > group admin role. Roles that mean something to the group. > > > Each Client Group would have some default roles defined. i.e. roles >> that allow a user to edit any client in the client group. >> >> >> I don't understand this >> >> > A Client Group could have a "client group admin" role. If a user has that > role it can manage clients in the group. Another role might be "client > membership admin". This role allows a user to add or remove clients from > the group. > > Conversely, user groups could have a "user group admin". When granted, > this role allows a user to manage users in the group. YOu can also do > things like define a "Manager" role for the group. This "Manager" would be > granted "user group admin" privileges and also granted access to other > systems like "HR", "Attendence", "Benefits", etc. > > I think this permission concept should be built into Keycloak as it is a > core feature. I'll write u a separate email about this. This is another reason why we need role namespaces. With a role namespace we can define internal roles that then don't end up conflicting with users own roles. For example as we have a role admin atm users can't define their own admin role and will have to name it differently. I think the fact that we have internal abstract clients to be able to create a namespace for internal admin roles speaks for itself. > > > > >> Each Client would have the same configuration options. They would be >> able to have an additional set of roles, permissions, scope, and >> overridable Protocol Policies. >> >> >> Same comment as above - why would a client have roles/permissions? I >> assume we where moving away from that with role namespaces >> >> > Again, I think role namespaces are redundant. > > A client can define a set of roles that it offers. A service account (the > client) can have roles associated with it so it can do certain actions. Some will want to have roles associated with a client (email-user), but others have organizational wide roles (manager, sales-guy, etc..). Role namespaces can deal with both, but client roles can't. > > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/0d8d5c42/attachment-0001.html From sthorger at redhat.com Thu Nov 5 13:53:46 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 19:53:46 +0100 Subject: [keycloak-dev] user group admins In-Reply-To: <563B9254.1090209@redhat.com> References: <563B9254.1090209@redhat.com> Message-ID: As I said in my previous email I think this is overusing and confusing the concept of a group. Users should be able use groups freely for their organizations without it conflicting with groups Keycloak uses to define permissions. As I proposed we could introduce the concept of an organization/domain. An admin would then have one or more roles associated with an domain. The organization/domain would simply be a namespace within the Keycloak namespace: org.keycloak//view-clients org.keycloak//manage-clients ... One issue with changing "permissions" on the admin endpoints is that currently we have a duplicated set of these as the master realm and each individual realm can have these. This is error prone, insecure and just outright confusing IMO. We should get rid of the master realm and simply have the admin endpoints and console hosted under a specific realm. This would also simplify the URLs for other things. So the URLs become: * /admin * /protocols * /... On 5 November 2015 at 18:31, Bill Burke wrote: > A few users have been asking about the ability to limit the admin to > managing a set of users in a group. > > I know Pedro is doing permission work, but IMO, this type of thing needs > to be integrated and natural to the Keycloak UI as it would be a > fundamental feature. > > Here's a proposal though based on my layout of User Groups in a previous > email. > > Group Admins need to be able to: > * Manage group membership > * Manage users within a group to do things like reset credentials and > other user management actions. > * Grant roles to users within a group > * Add attributes to the group > * Grant roles to the group > > So, if each User Group had its own Role Namespace, we could define one > or more roles that grant each of those permissions. i.e. > "group-membership-admin", "group-user-admin", "group-admin". If a User > Gruop has its own Role Namespace, it becomes really easy to compose > adminstrative access. So you could define individual "admin groups" and > grant users in those groups permission to manage one more groups. > > If groups don't have their own Role Namespace, you need a way to > associate a role to each group admin permission. > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/8fbc16db/attachment.html From sthorger at redhat.com Thu Nov 5 13:58:26 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 19:58:26 +0100 Subject: [keycloak-dev] controlling which roles an admin can grant In-Reply-To: <563B925B.9030908@redhat.com> References: <563B925B.9030908@redhat.com> Message-ID: Sounds complex and confusing to me. Also how do you specify how's allowed to manage the role granting permissions? A simpler approach would be to simply require an admin to have a role to be able to grant it to another user. When an admin creates a role they would be given that role as well. You an also composite roles to then achieve the same as you're mentioning above. On 5 November 2015 at 18:31, Bill Burke wrote: > One of things that we need to be able to do if we have the idea of a > "Group Admin" is to control specifically which role mappings an admin is > allowed to grant. One of the places this comes up currently is that if > an admin has the "manage-users" role, they can pretty much add any > permission they want to themselves and get access to the whole realm. > > IMO, this is something we need now. It needs to be built into our admin > UI. > > So, how could we add the ability to control which roles an admin is > allowed to grant? Under the "Roles" menu option there would be a "Grant > Permissions" tab. Here, the admin can select a role and specify a list > of roles that can be granted if a user has that role. > > Here's an example: > > Let's say there are 2 sales applications "reporting" and "analytics". > Each of the apps has defined an "admin" and "user" role. We want to have > a developer manage user access to these systems. > > 1. Define "Sales Access Control Manager" role. > 2. Go into "Roles" menu > 3. Go to the "Role Granting Permissions" tab. > 4. Select the "Sales Access Control Manager" role > 5. Select and add the "reporting.user", "reporting.admin", > "analytics.user", and "analytics.admin" roles to the list of roles a > "Sales Access Control Manager" is allowed to grant. > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/b2213059/attachment.html From ssilvert at redhat.com Thu Nov 5 14:24:48 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Thu, 05 Nov 2015 14:24:48 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: References: <5639253A.7090408@redhat.com> <563B611F.2070304@redhat.com> Message-ID: <563BAD00.5050609@redhat.com> We could do a lot worse than just following the basic RBAC design described on Wikipedia: https://en.wikipedia.org/wiki/Role-based_access_control Right now we're arguing over both definitions AND implementations. It's impossible to design this over email if we can't even settle on definitions. Therefore, I propose we just use the definitions in wikipedia. At least it's neutral. * S = Subject = A person or automated agent * R = Role = Job function or title which defines an authority level * P = Permissions = An approval of a mode of access to a resource * SE = Session = A mapping involving S, R and/or P * SA = Subject Assignment * PA = Permission Assignment * RH = Partially ordered Role Hierarchy. RH can also be written: ? (The notation: x ? y means that x inherits the permissions of y.) o A subject can have multiple roles. o A role can have multiple subjects. o A role can have many permissions. o A permission can be assigned to many roles. o An operation can be assigned many permissions. o A permission can be assigned to many operations. Note: In my mind, S = keycloak user, and SE = keycloak group. But whatever, as long as we agree on definitions we can then decide what flavor of RBAC to implement. On 11/5/2015 1:44 PM, Stian Thorgersen wrote: > > > On 5 November 2015 at 15:01, Bill Burke > wrote: > > > > On 11/5/2015 6:23 AM, Stian Thorgersen wrote: > > > > On 3 November 2015 at 22:20, Bill Burke > >> wrote: > > In my previous email I talked about combining Groups and Role > Namespaces. Now I want to talk about User Groups vs. > Client Groups. > > User Groups would manage a set of users. Members would > automatically > inherit a set of "permissions": a set of roles. User > Groups would also > provide a set of attributes that the user inherits. > > > Permission != role > > > I'd like to introduce the concept of a Client Group. > Client Group would > have: > > * Roles - basically a role namespace > > > -1 Having roles tied to a client or client group is exactly > what we > should go away from. IMO role namespaces should be a > completely separate > thing. > > > I don't agree at all. If User Groups and Client Groups exist, > there is no need for role namespaces. It is stupid to have to > create another concept (role namespace) to define the roles one > specific client or a group of clients expects. > > > I've never the concept of realm and client roles. It's been difficult > to explain and strange to use. I've always just used realm roles. It's > a strange and limiting concept. Introducing client groups with further > places to define roles just makes matters even worse. Now users have > two go 3 different places to define roles: > > * Realm > * Client Groups > * Clients > > What happens if a client group and a client both have the same role by > the way? > > It's a strange limitation. At least personally if I was using Keycloak > I would simply use realm roles alone and define my own hierarchy with > a delimiter. > > It's much better to have a single place to define roles, under the > roles tab. Then allow users can define the namespaces/hierarchy they want. > > Role namespaces are easier to deal with and at the same time more > flexible. > > I just don't see any reason why we would have roles specific to a > client or client group. > > > If you combine Role namespace and Groups you can define things > like a group admin role. Roles that mean something to the group. > > > Each Client Group would have some default roles defined. > i.e. roles > that allow a user to edit any client in the client group. > > > I don't understand this > > > A Client Group could have a "client group admin" role. If a user > has that role it can manage clients in the group. Another role > might be "client membership admin". This role allows a user to > add or remove clients from the group. > > Conversely, user groups could have a "user group admin". When > granted, this role allows a user to manage users in the group. > YOu can also do things like define a "Manager" role for the > group. This "Manager" would be granted "user group admin" > privileges and also granted access to other systems like "HR", > "Attendence", "Benefits", etc. > > I think this permission concept should be built into Keycloak as > it is a core feature. I'll write u a separate email about this. > > > This is another reason why we need role namespaces. With a role > namespace we can define internal roles that then don't end up > conflicting with users own roles. For example as we have a role admin > atm users can't define their own admin role and will have to name it > differently. > > I think the fact that we have internal abstract clients to be able to > create a namespace for internal admin roles speaks for itself. > > > > > > Each Client would have the same configuration options. > They would be > able to have an additional set of roles, permissions, > scope, and > overridable Protocol Policies. > > > Same comment as above - why would a client have > roles/permissions? I > assume we where moving away from that with role namespaces > > > Again, I think role namespaces are redundant. > > A client can define a set of roles that it offers. A service > account (the client) can have roles associated with it so it can > do certain actions. > > > Some will want to have roles associated with a client (email-user), > but others have organizational wide roles (manager, sales-guy, etc..). > Role namespaces can deal with both, but client roles can't. > > > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/6a23cc64/attachment-0001.html From sthorger at redhat.com Thu Nov 5 14:36:06 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 5 Nov 2015 20:36:06 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <563BAD00.5050609@redhat.com> References: <5639253A.7090408@redhat.com> <563B611F.2070304@redhat.com> <563BAD00.5050609@redhat.com> Message-ID: We're only providing parts of RBAC now. The complete picture is what Pedro is working on with his AuthZ service. >From the definitions above we're actually only providing S and R. SE is not a group as a group doesn't provide any permissions. I'm not 100% sure what the group would be in the above, but I would think it's just part of S. A group is simply a means of assigning a role to a group of users. On 5 November 2015 at 20:24, Stan Silvert wrote: > We could do a lot worse than just following the basic RBAC design > described on Wikipedia: > https://en.wikipedia.org/wiki/Role-based_access_control > > Right now we're arguing over both definitions AND implementations. It's > impossible to design this over email if we can't even settle on definitions. > > Therefore, I propose we just use the definitions in wikipedia. At least > it's neutral. > > - S = Subject = A person or automated agent > - R = Role = Job function or title which defines an authority level > - P = Permissions = An approval of a mode of access to a resource > - SE = Session = A mapping involving S, R and/or P > - SA = Subject Assignment > - PA = Permission Assignment > - RH = Partially ordered Role Hierarchy. RH can also be written: ? > (The notation: x ? y means that x inherits the permissions of y.) > - A subject can have multiple roles. > - A role can have multiple subjects. > - A role can have many permissions. > - A permission can be assigned to many roles. > - An operation can be assigned many permissions. > - A permission can be assigned to many operations. > > > Note: In my mind, S = keycloak user, and SE = keycloak group. But > whatever, as long as we agree on definitions we can then decide what flavor > of RBAC to implement. > > > On 11/5/2015 1:44 PM, Stian Thorgersen wrote: > > > > On 5 November 2015 at 15:01, Bill Burke wrote: > >> >> >> On 11/5/2015 6:23 AM, Stian Thorgersen wrote: >> >>> >>> >>> On 3 November 2015 at 22:20, Bill Burke >> > wrote: >>> >>> In my previous email I talked about combining Groups and Role >>> Namespaces. Now I want to talk about User Groups vs. Client Groups. >>> >>> User Groups would manage a set of users. Members would automatically >>> inherit a set of "permissions": a set of roles. User Groups would >>> also >>> provide a set of attributes that the user inherits. >>> >>> >>> Permission != role >>> >>> >>> I'd like to introduce the concept of a Client Group. Client Group >>> would >>> have: >>> >>> * Roles - basically a role namespace >>> >>> >>> -1 Having roles tied to a client or client group is exactly what we >>> should go away from. IMO role namespaces should be a completely separate >>> thing. >>> >>> >> I don't agree at all. If User Groups and Client Groups exist, there is >> no need for role namespaces. It is stupid to have to create another >> concept (role namespace) to define the roles one specific client or a group >> of clients expects. >> > > I've never the concept of realm and client roles. It's been difficult to > explain and strange to use. I've always just used realm roles. It's a > strange and limiting concept. Introducing client groups with further places > to define roles just makes matters even worse. Now users have two go 3 > different places to define roles: > > * Realm > * Client Groups > * Clients > > What happens if a client group and a client both have the same role by the > way? > > It's a strange limitation. At least personally if I was using Keycloak I > would simply use realm roles alone and define my own hierarchy with a > delimiter. > > It's much better to have a single place to define roles, under the roles > tab. Then allow users can define the namespaces/hierarchy they want. > > Role namespaces are easier to deal with and at the same time more > flexible. > > I just don't see any reason why we would have roles specific to a client > or client group. > > >> >> If you combine Role namespace and Groups you can define things like a >> group admin role. Roles that mean something to the group. >> >> >> Each Client Group would have some default roles defined. i.e. roles >>> that allow a user to edit any client in the client group. >>> >>> >>> I don't understand this >>> >>> >> A Client Group could have a "client group admin" role. If a user has >> that role it can manage clients in the group. Another role might be >> "client membership admin". This role allows a user to add or remove >> clients from the group. >> >> Conversely, user groups could have a "user group admin". When granted, >> this role allows a user to manage users in the group. YOu can also do >> things like define a "Manager" role for the group. This "Manager" would be >> granted "user group admin" privileges and also granted access to other >> systems like "HR", "Attendence", "Benefits", etc. >> >> I think this permission concept should be built into Keycloak as it is a >> core feature. I'll write u a separate email about this. > > > This is another reason why we need role namespaces. With a role namespace > we can define internal roles that then don't end up conflicting with users > own roles. For example as we have a role admin atm users can't define their > own admin role and will have to name it differently. > > I think the fact that we have internal abstract clients to be able to > create a namespace for internal admin roles speaks for itself. > > >> >> >> >> >>> Each Client would have the same configuration options. They would be >>> able to have an additional set of roles, permissions, scope, and >>> overridable Protocol Policies. >>> >>> >>> Same comment as above - why would a client have roles/permissions? I >>> assume we where moving away from that with role namespaces >>> >>> >> Again, I think role namespaces are redundant. >> >> A client can define a set of roles that it offers. A service account >> (the client) can have roles associated with it so it can do certain actions. > > > Some will want to have roles associated with a client (email-user), but > others have organizational wide roles (manager, sales-guy, etc..). Role > namespaces can deal with both, but client roles can't. > > >> >> >> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> > > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/964fa220/attachment.html From paul.wolf23 at googlemail.com Thu Nov 5 15:34:30 2015 From: paul.wolf23 at googlemail.com (Paul Wolf) Date: Thu, 5 Nov 2015 21:34:30 +0100 Subject: [keycloak-dev] Password grant flow with client consent is bugged In-Reply-To: <563B7AAE.4070403@redhat.com> References: <563B7AAE.4070403@redhat.com> Message-ID: Thanks for your answer, for the password grant flow i created: - a JIRA issue: https://issues.jboss.org/browse/KEYCLOAK-2038 - and a pull request: https://github.com/keycloak/keycloak/pull/1792 Is it ok like that? Can I expect the change for the comming release? Paul 2015-11-05 16:50 GMT+01:00 Marek Posolda : > On 05/11/15 14:43, Paul Wolf wrote: > > Hi guys, > > I am currently evaluating Keycloak and so far I have only two complaints: > 1. The implicit flow is not supported, but I already found a Jira issue for > that. How realistic is it that the feature comes with 1.7CR1? > > It's planned and very realistic that it really happens :-) > > 2. When the client consent is needed for an application, which uses the > password grant flow, the flow fails all the time... Even if the consent has > been given. This seems an easy to fix bug. Will you fix that? Should I write > a fix and make a pull request? > > +1 > > makes sense to me to let the password grant flow happen if consent has been > already given. Feel free to create JIRA and PR will be even better. Current > checking of consent is done in AuthenticationManager.actionRequired method. > I hope you can extract the consent checking to separate method and reuse it. > > Marek > > Otherwise good stuff so far :) > > Best regards, > Paul > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > From ssilvert at redhat.com Thu Nov 5 15:40:42 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Thu, 05 Nov 2015 15:40:42 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: References: <5639253A.7090408@redhat.com> <563B611F.2070304@redhat.com> <563BAD00.5050609@redhat.com> Message-ID: <563BBECA.8090509@redhat.com> On 11/5/2015 2:36 PM, Stian Thorgersen wrote: > We're only providing parts of RBAC now. The complete picture is what > Pedro is working on with his AuthZ service. Yea, as I understand it, Pedro is doing P. (P for Pedro!) And also, he's filling in the rest of the gaps surrounding P. > > From the definitions above we're actually only providing S and R. SE > is not a group as a group doesn't provide any permissions. Maybe that's a good reason to stick with the definitions below. I see "Group" as a way to implement the mapping called for in SE. But it doesn't have to be that way. > > I'm not 100% sure what the group would be in the above, but I would > think it's just part of S. A group is simply a means of assigning a > role to a group of users. > > > On 5 November 2015 at 20:24, Stan Silvert > wrote: > > We could do a lot worse than just following the basic RBAC design > described on Wikipedia: > https://en.wikipedia.org/wiki/Role-based_access_control > > Right now we're arguing over both definitions AND > implementations. It's impossible to design this over email if we > can't even settle on definitions. > > Therefore, I propose we just use the definitions in wikipedia. At > least it's neutral. > > * S = Subject = A person or automated agent > * R = Role = Job function or title which defines an authority level > * P = Permissions = An approval of a mode of access to a resource > * SE = Session = A mapping involving S, R and/or P > * SA = Subject Assignment > * PA = Permission Assignment > * RH = Partially ordered Role Hierarchy. RH can also be written: > ? (The notation: x ? y means that x inherits the permissions > of y.) > o A subject can have multiple roles. > o A role can have multiple subjects. > o A role can have many permissions. > o A permission can be assigned to many roles. > o An operation can be assigned many permissions. > o A permission can be assigned to many operations. > > > Note: In my mind, S = keycloak user, and SE = keycloak group. But > whatever, as long as we agree on definitions we can then decide > what flavor of RBAC to implement. > > > On 11/5/2015 1:44 PM, Stian Thorgersen wrote: >> >> >> On 5 November 2015 at 15:01, Bill Burke > > wrote: >> >> >> >> On 11/5/2015 6:23 AM, Stian Thorgersen wrote: >> >> >> >> On 3 November 2015 at 22:20, Bill Burke >> >> >> wrote: >> >> In my previous email I talked about combining Groups >> and Role >> Namespaces. Now I want to talk about User Groups vs. >> Client Groups. >> >> User Groups would manage a set of users. Members >> would automatically >> inherit a set of "permissions": a set of roles. User >> Groups would also >> provide a set of attributes that the user inherits. >> >> >> Permission != role >> >> >> I'd like to introduce the concept of a Client Group. >> Client Group would >> have: >> >> * Roles - basically a role namespace >> >> >> -1 Having roles tied to a client or client group is >> exactly what we >> should go away from. IMO role namespaces should be a >> completely separate >> thing. >> >> >> I don't agree at all. If User Groups and Client Groups >> exist, there is no need for role namespaces. It is stupid to >> have to create another concept (role namespace) to define the >> roles one specific client or a group of clients expects. >> >> >> I've never the concept of realm and client roles. It's been >> difficult to explain and strange to use. I've always just used >> realm roles. It's a strange and limiting concept. Introducing >> client groups with further places to define roles just makes >> matters even worse. Now users have two go 3 different places to >> define roles: >> >> * Realm >> * Client Groups >> * Clients >> >> What happens if a client group and a client both have the same >> role by the way? >> >> It's a strange limitation. At least personally if I was using >> Keycloak I would simply use realm roles alone and define my own >> hierarchy with a delimiter. >> >> It's much better to have a single place to define roles, under >> the roles tab. Then allow users can define the >> namespaces/hierarchy they want. >> >> Role namespaces are easier to deal with and at the same time more >> flexible. >> >> I just don't see any reason why we would have roles specific to a >> client or client group. >> >> >> If you combine Role namespace and Groups you can define >> things like a group admin role. Roles that mean something to >> the group. >> >> >> Each Client Group would have some default roles >> defined. i.e. roles >> that allow a user to edit any client in the client group. >> >> >> I don't understand this >> >> >> A Client Group could have a "client group admin" role. If a >> user has that role it can manage clients in the group. >> Another role might be "client membership admin". This role >> allows a user to add or remove clients from the group. >> >> Conversely, user groups could have a "user group admin". >> When granted, this role allows a user to manage users in the >> group. YOu can also do things like define a "Manager" role >> for the group. This "Manager" would be granted "user group >> admin" privileges and also granted access to other systems >> like "HR", "Attendence", "Benefits", etc. >> >> I think this permission concept should be built into Keycloak >> as it is a core feature. I'll write u a separate email about >> this. >> >> >> This is another reason why we need role namespaces. With a role >> namespace we can define internal roles that then don't end up >> conflicting with users own roles. For example as we have a role >> admin atm users can't define their own admin role and will have >> to name it differently. >> >> I think the fact that we have internal abstract clients to be >> able to create a namespace for internal admin roles speaks for >> itself. >> >> >> >> >> >> Each Client would have the same configuration >> options. They would be >> able to have an additional set of roles, permissions, >> scope, and >> overridable Protocol Policies. >> >> >> Same comment as above - why would a client have >> roles/permissions? I >> assume we where moving away from that with role namespaces >> >> >> Again, I think role namespaces are redundant. >> >> A client can define a set of roles that it offers. A service >> account (the client) can have roles associated with it so it >> can do certain actions. >> >> >> Some will want to have roles associated with a client >> (email-user), but others have organizational wide roles (manager, >> sales-guy, etc..). Role namespaces can deal with both, but client >> roles can't. >> >> >> >> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151105/7f5c9747/attachment-0001.html From bburke at redhat.com Thu Nov 5 15:58:16 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 5 Nov 2015 15:58:16 -0500 Subject: [keycloak-dev] controlling which roles an admin can grant In-Reply-To: References: <563B925B.9030908@redhat.com> Message-ID: <563BC2E8.9010505@redhat.com> On 11/5/2015 1:58 PM, Stian Thorgersen wrote: > Sounds complex and confusing to me. Also how do you specify how's > allowed to manage the role granting permissions? > My proposal is *simpler* and very explicit. All this is is assigning admin permissions to a role. There would be a realm-wide role for admins that are allowed to set up role granting permissions. Just like we have for view-user, etc. So, the master admin sets up the role granting permissions, then assigns role granting roles to each subset of "junior" admins. > A simpler approach would be to simply require an admin to have a role to > be able to grant it to another user. When an admin creates a role they > would be given that role as well. You an also composite roles to then > achieve the same as you're mentioning above. > I started with that approach, but I thought it was too implicit and confusing. There will be cases where a user has admin permissions for a client, but you don't want to allow them to grant this permission to others. Its like contributors at git hub. Contributors can merge PRs, but they can't grant others contributor access. > On 5 November 2015 at 18:31, Bill Burke > wrote: > > One of things that we need to be able to do if we have the idea of a > "Group Admin" is to control specifically which role mappings an admin is > allowed to grant. One of the places this comes up currently is that if > an admin has the "manage-users" role, they can pretty much add any > permission they want to themselves and get access to the whole realm. > > IMO, this is something we need now. It needs to be built into our > admin UI. > > So, how could we add the ability to control which roles an admin is > allowed to grant? Under the "Roles" menu option there would be a "Grant > Permissions" tab. Here, the admin can select a role and specify a list > of roles that can be granted if a user has that role. > > Here's an example: > > Let's say there are 2 sales applications "reporting" and "analytics". > Each of the apps has defined an "admin" and "user" role. We want to have > a developer manage user access to these systems. > > 1. Define "Sales Access Control Manager" role. > 2. Go into "Roles" menu > 3. Go to the "Role Granting Permissions" tab. > 4. Select the "Sales Access Control Manager" role > 5. Select and add the "reporting.user", "reporting.admin", > "analytics.user", and "analytics.admin" roles to the list of roles a > "Sales Access Control Manager" is allowed to grant. > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Thu Nov 5 16:51:02 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 5 Nov 2015 16:51:02 -0500 Subject: [keycloak-dev] user group admins In-Reply-To: References: <563B9254.1090209@redhat.com> Message-ID: <563BCF46.3070005@redhat.com> On 11/5/2015 1:53 PM, Stian Thorgersen wrote: > As I said in my previous email I think this is overusing and confusing > the concept of a group. > I don't think it is confusing, but it may be overloading the concept of a group. > Users should be able use groups freely for their organizations without > it conflicting with groups Keycloak uses to define permissions. > > As I proposed we could introduce the concept of an organization/domain. > An admin would then have one or more roles associated with an domain. > The organization/domain would simply be a namespace within the Keycloak > namespace: > > org.keycloak//view-clients > org.keycloak//manage-clients > ... > An organization is just a namespace for roles and clients? As well as a subset of realm users? I like clients to be able to have their own role namespace as you have a very clear definition of how the client is defined. This client lives at this URL, has these mappers, and publishes/provides these roles and permissions. Same for User Groups: This "User Group" has a set of default permissions. As well as a defined set of roles. An "employee" or "manager" in "Accounting" would have different permissions than a "employee" or "manager" within "Engineering". With separate role namespaces you have no clear idea what roles are important to which clients and users. > One issue with changing "permissions" on the admin endpoints is that > currently we have a duplicated set of these as the master realm and each > individual realm can have these. This is error prone, insecure and just > outright confusing IMO. We should get rid of the master realm and simply > have the admin endpoints and console hosted under a specific realm. This > would also simplify the URLs for other things. So the URLs become: > Without a master realm, you have to figure out the chicken and egg problem. How do you create a new realm? How do you manage that new realm? How can you manage more than one realm with one login? Red Hat IT has a need for two separate realms. One for employees and one for customers. Will they have to have two separate admin accounts on each realm to manage them? > * /admin > * /protocols > * /... > Our URL scheme is already like this. You can log into a specific admin console for a specific realm. The "issue" (depending on your perspective) is that the admin REST interface allows you to interact with any realm. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Fri Nov 6 01:32:39 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 07:32:39 +0100 Subject: [keycloak-dev] controlling which roles an admin can grant In-Reply-To: <563BC2E8.9010505@redhat.com> References: <563B925B.9030908@redhat.com> <563BC2E8.9010505@redhat.com> Message-ID: I don't agree that it's simpler, but I agree with your other points. I'm not to worried about the simplicity as I assume this won't be used by most people, only those that need to limit what roles specific admins can grant. So we need a role to allow granting all roles and a role to manage role granting permissions? This further speaks to us introducing role namespaces as then users can define the role granting permissions roles (hm.. simpler did you say?) in a separate namespace. BTW in the future with the authz services hopefully users can define their own policies for securing the admin endpoints. We'd ship with some default policies and permissions, then users can change that however they want. The admin endpoints would just be secured by the authz services, rather than our bespoke code we use ATM. On 5 November 2015 at 21:58, Bill Burke wrote: > > > On 11/5/2015 1:58 PM, Stian Thorgersen wrote: > >> Sounds complex and confusing to me. Also how do you specify how's >> allowed to manage the role granting permissions? >> >> > My proposal is *simpler* and very explicit. All this is is assigning > admin permissions to a role. > > > > There would be a realm-wide role for admins that are allowed to set up > role granting permissions. Just like we have for view-user, etc. So, the > master admin sets up the role granting permissions, then assigns role > granting roles to each subset of "junior" admins. > > > A simpler approach would be to simply require an admin to have a role to >> be able to grant it to another user. When an admin creates a role they >> would be given that role as well. You an also composite roles to then >> achieve the same as you're mentioning above. >> >> > I started with that approach, but I thought it was too implicit and > confusing. There will be cases where a user has admin permissions for a > client, but you don't want to allow them to grant this permission to > others. Its like contributors at git hub. Contributors can merge PRs, but > they can't grant others contributor access. > > > > > On 5 November 2015 at 18:31, Bill Burke > > wrote: >> >> One of things that we need to be able to do if we have the idea of a >> "Group Admin" is to control specifically which role mappings an admin >> is >> allowed to grant. One of the places this comes up currently is that >> if >> an admin has the "manage-users" role, they can pretty much add any >> permission they want to themselves and get access to the whole realm. >> >> IMO, this is something we need now. It needs to be built into our >> admin UI. >> >> So, how could we add the ability to control which roles an admin is >> allowed to grant? Under the "Roles" menu option there would be a >> "Grant >> Permissions" tab. Here, the admin can select a role and specify a >> list >> of roles that can be granted if a user has that role. >> >> Here's an example: >> >> Let's say there are 2 sales applications "reporting" and "analytics". >> Each of the apps has defined an "admin" and "user" role. We want to >> have >> a developer manage user access to these systems. >> >> 1. Define "Sales Access Control Manager" role. >> 2. Go into "Roles" menu >> 3. Go to the "Role Granting Permissions" tab. >> 4. Select the "Sales Access Control Manager" role >> 5. Select and add the "reporting.user", "reporting.admin", >> "analytics.user", and "analytics.admin" roles to the list of roles a >> "Sales Access Control Manager" is allowed to grant. >> >> > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/c7a1a40a/attachment.html From sthorger at redhat.com Fri Nov 6 01:46:48 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 07:46:48 +0100 Subject: [keycloak-dev] user group admins In-Reply-To: <563BCF46.3070005@redhat.com> References: <563B9254.1090209@redhat.com> <563BCF46.3070005@redhat.com> Message-ID: On 5 November 2015 at 22:51, Bill Burke wrote: > > > On 11/5/2015 1:53 PM, Stian Thorgersen wrote: > >> As I said in my previous email I think this is overusing and confusing >> the concept of a group. >> >> > I don't think it is confusing, but it may be overloading the concept of a > group. > > Users should be able use groups freely for their organizations without >> it conflicting with groups Keycloak uses to define permissions. >> >> As I proposed we could introduce the concept of an organization/domain. >> An admin would then have one or more roles associated with an domain. >> The organization/domain would simply be a namespace within the Keycloak >> namespace: >> >> org.keycloak//view-clients >> org.keycloak//manage-clients >> ... >> >> > An organization is just a namespace for roles and clients? As well as a > subset of realm users? > > I like clients to be able to have their own role namespace as you have a > very clear definition of how the client is defined. This client lives at > this URL, has these mappers, and publishes/provides these roles and > permissions. > > Same for User Groups: This "User Group" has a set of default > permissions. As well as a defined set of roles. An "employee" or > "manager" in "Accounting" would have different permissions than a > "employee" or "manager" within "Engineering". > > With separate role namespaces you have no clear idea what roles are > important to which clients and users. As I said some use organizational wide roles, others have roles specific to a client or a group of clients. What you are proposing only works with those that have roles specific to a client. Namespaces works for both. I disagree that it's a big problem with separate role namespaces as users can easily define sensible namespaces that ties a group of clients to a namespace. We could even have a role namespace attribute on a client group. > > > > One issue with changing "permissions" on the admin endpoints is that >> currently we have a duplicated set of these as the master realm and each >> individual realm can have these. This is error prone, insecure and just >> outright confusing IMO. We should get rid of the master realm and simply >> have the admin endpoints and console hosted under a specific realm. This >> would also simplify the URLs for other things. So the URLs become: >> >> > Without a master realm, you have to figure out the chicken and egg > problem. How do you create a new realm? How do you manage that new > realm? How can you manage more than one realm with one login? Red Hat IT > has a need for two separate realms. One for employees and one for > customers. Will they have to have two separate admin accounts on each > realm to manage them? Yes, we do need to figure that out - but I don't think it's a problem and I'm confident we can find a decent solution to it. We could probably implement it so that we'd still have the same realm drop-down in the admin console. Permissions would be done something along the lines off: * An option on a realm that permits admins in it to create new realms * Use identity brokering to provide SSO to manage other realms - this could be an idp that's hidden and can only be used through the idp_hint However, I'm not sure if it's a big deal that an admin has to login to the individual realms to manage them. > > > * /admin >> * /protocols >> * /... >> >> > Our URL scheme is already like this. You can log into a specific admin > console for a specific realm. The "issue" (depending on your perspective) > is that the admin REST interface allows you to interact with any realm. Our URLs at the moment are prefixed with '/realms', then there's the two separate URLs for admin endpoints. Several users has been confused about the admin client libraries where you login with one realm and manage another. I still get confused when dealing with the admin endpoints and dealing with permissions. What is the master realm client, what is the non-master realm client, etc... > > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/36e4a9f3/attachment-0001.html From sthorger at redhat.com Fri Nov 6 01:48:40 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 07:48:40 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <563BBECA.8090509@redhat.com> References: <5639253A.7090408@redhat.com> <563B611F.2070304@redhat.com> <563BAD00.5050609@redhat.com> <563BBECA.8090509@redhat.com> Message-ID: On 5 November 2015 at 21:40, Stan Silvert wrote: > On 11/5/2015 2:36 PM, Stian Thorgersen wrote: > > We're only providing parts of RBAC now. The complete picture is what Pedro > is working on with his AuthZ service. > > Yea, as I understand it, Pedro is doing P. (P for Pedro!) And also, he's > filling in the rest of the gaps surrounding P. > > > From the definitions above we're actually only providing S and R. SE is > not a group as a group doesn't provide any permissions. > > Maybe that's a good reason to stick with the definitions below. I see > "Group" as a way to implement the mapping called for in SE. But it doesn't > have to be that way. > Group is not SE, I'm pretty sure of that. A group is just an "attribute" of the subject. It doesn't "provide" any permissions or any mapping between subjects and permissions. > > > I'm not 100% sure what the group would be in the above, but I would think > it's just part of S. A group is simply a means of assigning a role to a > group of users. > > > On 5 November 2015 at 20:24, Stan Silvert wrote: > >> We could do a lot worse than just following the basic RBAC design >> described on Wikipedia: >> https://en.wikipedia.org/wiki/Role-based_access_control >> >> Right now we're arguing over both definitions AND implementations. It's >> impossible to design this over email if we can't even settle on definitions. >> >> Therefore, I propose we just use the definitions in wikipedia. At least >> it's neutral. >> >> - S = Subject = A person or automated agent >> - R = Role = Job function or title which defines an authority level >> - P = Permissions = An approval of a mode of access to a resource >> - SE = Session = A mapping involving S, R and/or P >> - SA = Subject Assignment >> - PA = Permission Assignment >> - RH = Partially ordered Role Hierarchy. RH can also be written: ? >> (The notation: x ? y means that x inherits the permissions of y.) >> - A subject can have multiple roles. >> - A role can have multiple subjects. >> - A role can have many permissions. >> - A permission can be assigned to many roles. >> - An operation can be assigned many permissions. >> - A permission can be assigned to many operations. >> >> >> Note: In my mind, S = keycloak user, and SE = keycloak group. But >> whatever, as long as we agree on definitions we can then decide what flavor >> of RBAC to implement. >> >> >> On 11/5/2015 1:44 PM, Stian Thorgersen wrote: >> >> >> >> On 5 November 2015 at 15:01, Bill Burke wrote: >> >>> >>> >>> On 11/5/2015 6:23 AM, Stian Thorgersen wrote: >>> >>>> >>>> >>>> On 3 November 2015 at 22:20, Bill Burke >>> > wrote: >>>> >>>> In my previous email I talked about combining Groups and Role >>>> Namespaces. Now I want to talk about User Groups vs. Client Groups. >>>> >>>> User Groups would manage a set of users. Members would >>>> automatically >>>> inherit a set of "permissions": a set of roles. User Groups would >>>> also >>>> provide a set of attributes that the user inherits. >>>> >>>> >>>> Permission != role >>>> >>>> >>>> I'd like to introduce the concept of a Client Group. Client Group >>>> would >>>> have: >>>> >>>> * Roles - basically a role namespace >>>> >>>> >>>> -1 Having roles tied to a client or client group is exactly what we >>>> should go away from. IMO role namespaces should be a completely separate >>>> thing. >>>> >>>> >>> I don't agree at all. If User Groups and Client Groups exist, there is >>> no need for role namespaces. It is stupid to have to create another >>> concept (role namespace) to define the roles one specific client or a group >>> of clients expects. >>> >> >> I've never the concept of realm and client roles. It's been difficult to >> explain and strange to use. I've always just used realm roles. It's a >> strange and limiting concept. Introducing client groups with further places >> to define roles just makes matters even worse. Now users have two go 3 >> different places to define roles: >> >> * Realm >> * Client Groups >> * Clients >> >> What happens if a client group and a client both have the same role by >> the way? >> >> It's a strange limitation. At least personally if I was using Keycloak I >> would simply use realm roles alone and define my own hierarchy with a >> delimiter. >> >> It's much better to have a single place to define roles, under the roles >> tab. Then allow users can define the namespaces/hierarchy they want. >> >> Role namespaces are easier to deal with and at the same time more >> flexible. >> >> I just don't see any reason why we would have roles specific to a client >> or client group. >> >> >>> >>> If you combine Role namespace and Groups you can define things like a >>> group admin role. Roles that mean something to the group. >>> >>> >>> Each Client Group would have some default roles defined. i.e. roles >>>> that allow a user to edit any client in the client group. >>>> >>>> >>>> I don't understand this >>>> >>>> >>> A Client Group could have a "client group admin" role. If a user has >>> that role it can manage clients in the group. Another role might be >>> "client membership admin". This role allows a user to add or remove >>> clients from the group. >>> >>> Conversely, user groups could have a "user group admin". When granted, >>> this role allows a user to manage users in the group. YOu can also do >>> things like define a "Manager" role for the group. This "Manager" would be >>> granted "user group admin" privileges and also granted access to other >>> systems like "HR", "Attendence", "Benefits", etc. >>> >>> I think this permission concept should be built into Keycloak as it is a >>> core feature. I'll write u a separate email about this. >> >> >> This is another reason why we need role namespaces. With a role namespace >> we can define internal roles that then don't end up conflicting with users >> own roles. For example as we have a role admin atm users can't define their >> own admin role and will have to name it differently. >> >> I think the fact that we have internal abstract clients to be able to >> create a namespace for internal admin roles speaks for itself. >> >> >>> >>> >>> >>> >>>> Each Client would have the same configuration options. They would >>>> be >>>> able to have an additional set of roles, permissions, scope, and >>>> overridable Protocol Policies. >>>> >>>> >>>> Same comment as above - why would a client have roles/permissions? I >>>> assume we where moving away from that with role namespaces >>>> >>>> >>> Again, I think role namespaces are redundant. >>> >>> A client can define a set of roles that it offers. A service account >>> (the client) can have roles associated with it so it can do certain actions. >> >> >> Some will want to have roles associated with a client (email-user), but >> others have organizational wide roles (manager, sales-guy, etc..). Role >> namespaces can deal with both, but client roles can't. >> >> >>> >>> >>> >>> >>> -- >>> Bill Burke >>> JBoss, a division of Red Hat >>> http://bill.burkecentral.com >>> >> >> >> >> _______________________________________________ >> keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/59a98578/attachment.html From thomas.raehalme at aitiofinland.com Fri Nov 6 02:16:44 2015 From: thomas.raehalme at aitiofinland.com (Thomas Raehalme) Date: Fri, 6 Nov 2015 09:16:44 +0200 Subject: [keycloak-dev] adapter-core and errors In-Reply-To: <563B7777.8080906@redhat.com> References: <563B7777.8080906@redhat.com> Message-ID: I added https://issues.jboss.org/browse/KEYCLOAK-2041 for this issue. On Thu, Nov 5, 2015 at 5:36 PM, Bill Burke wrote: > Log a JIRA please. This is something I saw when doing SAML adapter and > should make it uniform across all adapters. > > On 11/5/2015 9:01 AM, Thomas Raehalme wrote: > > Hi, > > > > I'm playing around with client-side error pages regarding Keycloak > > authentication. > > > > I have an Java EE web application utilizing error-page definitions in > > web.xml. If authentication fails for a reason or another, shouldn't the > > adapter-core enable the use of these error-pages? > > > > At the moment all or most of the AuthChallenge implementations in > > adapter-core call the setStatus method of the HttpFacade. This results > > in blank responses with only the status being set. If sendError was used > > instead then the default error-page handling would kick in. > > > > Is there a reason I am missing why using setStatus instead of sendError > > in, for example, OAuthRequestAuthenticator? > > > > Best regards, > > Thomas > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- *Thomas Raehalme* *CTO, teknologiajohtaja* Mobile +358 40 545 0605 *Aitio Finland Oy* V?in?nkatu 26 A 40100 JYV?SKYL?, Finland Tel. +358 10 322 0040 www.aitiofinland.com *Codecenter on nyt Aitio ? me kun ei vain koodata!* -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/ecfd5477/attachment-0001.html From psilva at redhat.com Fri Nov 6 07:12:11 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Fri, 6 Nov 2015 07:12:11 -0500 (EST) Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> Message-ID: <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> Hi, Is keycloak-adapter-feature-pack the right pack to use to build distributions to deploy client applications only, which require only the adapters ? In this case, WFLY ones ... Regards. Pedro Igor From ssilvert at redhat.com Fri Nov 6 07:31:37 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Fri, 06 Nov 2015 07:31:37 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: References: <5639253A.7090408@redhat.com> <563B611F.2070304@redhat.com> <563BAD00.5050609@redhat.com> <563BBECA.8090509@redhat.com> Message-ID: <563C9DA9.7090704@redhat.com> On 11/6/2015 1:48 AM, Stian Thorgersen wrote: > > > On 5 November 2015 at 21:40, Stan Silvert > wrote: > > On 11/5/2015 2:36 PM, Stian Thorgersen wrote: >> We're only providing parts of RBAC now. The complete picture is >> what Pedro is working on with his AuthZ service. > Yea, as I understand it, Pedro is doing P. (P for Pedro!) And > also, he's filling in the rest of the gaps surrounding P. >> >> From the definitions above we're actually only providing S and R. >> SE is not a group as a group doesn't provide any permissions. > Maybe that's a good reason to stick with the definitions below. I > see "Group" as a way to implement the mapping called for in SE. > But it doesn't have to be that way. > > > Group is not SE, I'm pretty sure of that. A group is just an > "attribute" of the subject. It doesn't "provide" any permissions or > any mapping between subjects and permissions. Let's say you allow Permissions and Roles to be assigned to a Group. And you also design it such that any Subject who becomes a member of the group also gets the assigned Permissions and Roles. In that case, you have implemented a Group that acts as the SE. > > >> >> I'm not 100% sure what the group would be in the above, but I >> would think it's just part of S. A group is simply a means of >> assigning a role to a group of users. >> >> >> On 5 November 2015 at 20:24, Stan Silvert > > wrote: >> >> We could do a lot worse than just following the basic RBAC >> design described on Wikipedia: >> https://en.wikipedia.org/wiki/Role-based_access_control >> >> Right now we're arguing over both definitions AND >> implementations. It's impossible to design this over email >> if we can't even settle on definitions. >> >> Therefore, I propose we just use the definitions in >> wikipedia. At least it's neutral. >> >> * S = Subject = A person or automated agent >> * R = Role = Job function or title which defines an >> authority level >> * P = Permissions = An approval of a mode of access to a >> resource >> * SE = Session = A mapping involving S, R and/or P >> * SA = Subject Assignment >> * PA = Permission Assignment >> * RH = Partially ordered Role Hierarchy. RH can also be >> written: ? (The notation: x ? y means that x inherits the >> permissions of y.) >> o A subject can have multiple roles. >> o A role can have multiple subjects. >> o A role can have many permissions. >> o A permission can be assigned to many roles. >> o An operation can be assigned many permissions. >> o A permission can be assigned to many operations. >> >> >> Note: In my mind, S = keycloak user, and SE = keycloak >> group. But whatever, as long as we agree on definitions we >> can then decide what flavor of RBAC to implement. >> >> >> On 11/5/2015 1:44 PM, Stian Thorgersen wrote: >>> >>> >>> On 5 November 2015 at 15:01, Bill Burke >> > wrote: >>> >>> >>> >>> On 11/5/2015 6:23 AM, Stian Thorgersen wrote: >>> >>> >>> >>> On 3 November 2015 at 22:20, Bill Burke >>> >>> >> >> wrote: >>> >>> In my previous email I talked about combining >>> Groups and Role >>> Namespaces. Now I want to talk about User >>> Groups vs. Client Groups. >>> >>> User Groups would manage a set of users. Members >>> would automatically >>> inherit a set of "permissions": a set of roles. >>> User Groups would also >>> provide a set of attributes that the user inherits. >>> >>> >>> Permission != role >>> >>> >>> I'd like to introduce the concept of a Client >>> Group. Client Group would >>> have: >>> >>> * Roles - basically a role namespace >>> >>> >>> -1 Having roles tied to a client or client group is >>> exactly what we >>> should go away from. IMO role namespaces should be a >>> completely separate >>> thing. >>> >>> >>> I don't agree at all. If User Groups and Client Groups >>> exist, there is no need for role namespaces. It is >>> stupid to have to create another concept (role >>> namespace) to define the roles one specific client or a >>> group of clients expects. >>> >>> >>> I've never the concept of realm and client roles. It's been >>> difficult to explain and strange to use. I've always just >>> used realm roles. It's a strange and limiting concept. >>> Introducing client groups with further places to define >>> roles just makes matters even worse. Now users have two go 3 >>> different places to define roles: >>> >>> * Realm >>> * Client Groups >>> * Clients >>> >>> What happens if a client group and a client both have the >>> same role by the way? >>> >>> It's a strange limitation. At least personally if I was >>> using Keycloak I would simply use realm roles alone and >>> define my own hierarchy with a delimiter. >>> >>> It's much better to have a single place to define roles, >>> under the roles tab. Then allow users can define the >>> namespaces/hierarchy they want. >>> >>> Role namespaces are easier to deal with and at the same time >>> more flexible. >>> >>> I just don't see any reason why we would have roles specific >>> to a client or client group. >>> >>> >>> If you combine Role namespace and Groups you can define >>> things like a group admin role. Roles that mean >>> something to the group. >>> >>> >>> Each Client Group would have some default roles >>> defined. i.e. roles >>> that allow a user to edit any client in the >>> client group. >>> >>> >>> I don't understand this >>> >>> >>> A Client Group could have a "client group admin" role. >>> If a user has that role it can manage clients in the >>> group. Another role might be "client membership >>> admin". This role allows a user to add or remove >>> clients from the group. >>> >>> Conversely, user groups could have a "user group >>> admin". When granted, this role allows a user to manage >>> users in the group. YOu can also do things like define >>> a "Manager" role for the group. This "Manager" would be >>> granted "user group admin" privileges and also granted >>> access to other systems like "HR", "Attendence", >>> "Benefits", etc. >>> >>> I think this permission concept should be built into >>> Keycloak as it is a core feature. I'll write u a >>> separate email about this. >>> >>> >>> This is another reason why we need role namespaces. With a >>> role namespace we can define internal roles that then don't >>> end up conflicting with users own roles. For example as we >>> have a role admin atm users can't define their own admin >>> role and will have to name it differently. >>> >>> I think the fact that we have internal abstract clients to >>> be able to create a namespace for internal admin roles >>> speaks for itself. >>> >>> >>> >>> >>> >>> Each Client would have the same configuration >>> options. They would be >>> able to have an additional set of roles, >>> permissions, scope, and >>> overridable Protocol Policies. >>> >>> >>> Same comment as above - why would a client have >>> roles/permissions? I >>> assume we where moving away from that with role >>> namespaces >>> >>> >>> Again, I think role namespaces are redundant. >>> >>> A client can define a set of roles that it offers. A >>> service account (the client) can have roles associated >>> with it so it can do certain actions. >>> >>> >>> Some will want to have roles associated with a client >>> (email-user), but others have organizational wide roles >>> (manager, sales-guy, etc..). Role namespaces can deal with >>> both, but client roles can't. >>> >>> >>> >>> >>> >>> -- >>> Bill Burke >>> JBoss, a division of Red Hat >>> http://bill.burkecentral.com >>> >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/3c46df15/attachment-0001.html From sthorger at redhat.com Fri Nov 6 07:52:40 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 13:52:40 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <563C9DA9.7090704@redhat.com> References: <5639253A.7090408@redhat.com> <563B611F.2070304@redhat.com> <563BAD00.5050609@redhat.com> <563BBECA.8090509@redhat.com> <563C9DA9.7090704@redhat.com> Message-ID: I disagree. Firstly we're not adding Permissions now. Secondly the Group is either part of the Subject or it's the equivalent of a Role, that depends on policies (which are SE), not the mapping of Permissions to the Subject. Following your argument if you had allowed assigning permissions based on dob then the dob of a user would be an SE. On 6 November 2015 at 13:31, Stan Silvert wrote: > On 11/6/2015 1:48 AM, Stian Thorgersen wrote: > > > > On 5 November 2015 at 21:40, Stan Silvert wrote: > >> On 11/5/2015 2:36 PM, Stian Thorgersen wrote: >> >> We're only providing parts of RBAC now. The complete picture is what >> Pedro is working on with his AuthZ service. >> >> Yea, as I understand it, Pedro is doing P. (P for Pedro!) And also, >> he's filling in the rest of the gaps surrounding P. >> >> >> From the definitions above we're actually only providing S and R. SE is >> not a group as a group doesn't provide any permissions. >> >> Maybe that's a good reason to stick with the definitions below. I see >> "Group" as a way to implement the mapping called for in SE. But it doesn't >> have to be that way. >> > > Group is not SE, I'm pretty sure of that. A group is just an "attribute" > of the subject. It doesn't "provide" any permissions or any mapping between > subjects and permissions. > > Let's say you allow Permissions and Roles to be assigned to a Group. And > you also design it such that any Subject who becomes a member of the group > also gets the assigned Permissions and Roles. In that case, you have > implemented a Group that acts as the SE. > > > >> >> >> I'm not 100% sure what the group would be in the above, but I would think >> it's just part of S. A group is simply a means of assigning a role to a >> group of users. >> >> >> On 5 November 2015 at 20:24, Stan Silvert wrote: >> >>> We could do a lot worse than just following the basic RBAC design >>> described on Wikipedia: >>> https://en.wikipedia.org/wiki/Role-based_access_control >>> >>> Right now we're arguing over both definitions AND implementations. It's >>> impossible to design this over email if we can't even settle on definitions. >>> >>> Therefore, I propose we just use the definitions in wikipedia. At least >>> it's neutral. >>> >>> - S = Subject = A person or automated agent >>> - R = Role = Job function or title which defines an authority level >>> - P = Permissions = An approval of a mode of access to a resource >>> - SE = Session = A mapping involving S, R and/or P >>> - SA = Subject Assignment >>> - PA = Permission Assignment >>> - RH = Partially ordered Role Hierarchy. RH can also be written: ? >>> (The notation: x ? y means that x inherits the permissions of y.) >>> - A subject can have multiple roles. >>> - A role can have multiple subjects. >>> - A role can have many permissions. >>> - A permission can be assigned to many roles. >>> - An operation can be assigned many permissions. >>> - A permission can be assigned to many operations. >>> >>> >>> Note: In my mind, S = keycloak user, and SE = keycloak group. But >>> whatever, as long as we agree on definitions we can then decide what flavor >>> of RBAC to implement. >>> >>> >>> On 11/5/2015 1:44 PM, Stian Thorgersen wrote: >>> >>> >>> >>> On 5 November 2015 at 15:01, Bill Burke wrote: >>> >>>> >>>> >>>> On 11/5/2015 6:23 AM, Stian Thorgersen wrote: >>>> >>>>> >>>>> >>>>> On 3 November 2015 at 22:20, Bill Burke >>>> > wrote: >>>>> >>>>> In my previous email I talked about combining Groups and Role >>>>> Namespaces. Now I want to talk about User Groups vs. Client >>>>> Groups. >>>>> >>>>> User Groups would manage a set of users. Members would >>>>> automatically >>>>> inherit a set of "permissions": a set of roles. User Groups would >>>>> also >>>>> provide a set of attributes that the user inherits. >>>>> >>>>> >>>>> Permission != role >>>>> >>>>> >>>>> I'd like to introduce the concept of a Client Group. Client Group >>>>> would >>>>> have: >>>>> >>>>> * Roles - basically a role namespace >>>>> >>>>> >>>>> -1 Having roles tied to a client or client group is exactly what we >>>>> should go away from. IMO role namespaces should be a completely >>>>> separate >>>>> thing. >>>>> >>>>> >>>> I don't agree at all. If User Groups and Client Groups exist, there is >>>> no need for role namespaces. It is stupid to have to create another >>>> concept (role namespace) to define the roles one specific client or a group >>>> of clients expects. >>>> >>> >>> I've never the concept of realm and client roles. It's been difficult to >>> explain and strange to use. I've always just used realm roles. It's a >>> strange and limiting concept. Introducing client groups with further places >>> to define roles just makes matters even worse. Now users have two go 3 >>> different places to define roles: >>> >>> * Realm >>> * Client Groups >>> * Clients >>> >>> What happens if a client group and a client both have the same role by >>> the way? >>> >>> It's a strange limitation. At least personally if I was using Keycloak I >>> would simply use realm roles alone and define my own hierarchy with a >>> delimiter. >>> >>> It's much better to have a single place to define roles, under the roles >>> tab. Then allow users can define the namespaces/hierarchy they want. >>> >>> Role namespaces are easier to deal with and at the same time more >>> flexible. >>> >>> I just don't see any reason why we would have roles specific to a client >>> or client group. >>> >>> >>>> >>>> If you combine Role namespace and Groups you can define things like a >>>> group admin role. Roles that mean something to the group. >>>> >>>> >>>> Each Client Group would have some default roles defined. i.e. roles >>>>> that allow a user to edit any client in the client group. >>>>> >>>>> >>>>> I don't understand this >>>>> >>>>> >>>> A Client Group could have a "client group admin" role. If a user has >>>> that role it can manage clients in the group. Another role might be >>>> "client membership admin". This role allows a user to add or remove >>>> clients from the group. >>>> >>>> Conversely, user groups could have a "user group admin". When granted, >>>> this role allows a user to manage users in the group. YOu can also do >>>> things like define a "Manager" role for the group. This "Manager" would be >>>> granted "user group admin" privileges and also granted access to other >>>> systems like "HR", "Attendence", "Benefits", etc. >>>> >>>> I think this permission concept should be built into Keycloak as it is >>>> a core feature. I'll write u a separate email about this. >>> >>> >>> This is another reason why we need role namespaces. With a role >>> namespace we can define internal roles that then don't end up conflicting >>> with users own roles. For example as we have a role admin atm users can't >>> define their own admin role and will have to name it differently. >>> >>> I think the fact that we have internal abstract clients to be able to >>> create a namespace for internal admin roles speaks for itself. >>> >>> >>>> >>>> >>>> >>>> >>>>> Each Client would have the same configuration options. They would >>>>> be >>>>> able to have an additional set of roles, permissions, scope, and >>>>> overridable Protocol Policies. >>>>> >>>>> >>>>> Same comment as above - why would a client have roles/permissions? I >>>>> assume we where moving away from that with role namespaces >>>>> >>>>> >>>> Again, I think role namespaces are redundant. >>>> >>>> A client can define a set of roles that it offers. A service account >>>> (the client) can have roles associated with it so it can do certain actions. >>> >>> >>> Some will want to have roles associated with a client (email-user), but >>> others have organizational wide roles (manager, sales-guy, etc..). Role >>> namespaces can deal with both, but client roles can't. >>> >>> >>>> >>>> >>>> >>>> >>>> -- >>>> Bill Burke >>>> JBoss, a division of Red Hat >>>> http://bill.burkecentral.com >>>> >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/9c33f141/attachment-0001.html From sthorger at redhat.com Fri Nov 6 07:53:45 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 13:53:45 +0100 Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> References: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> Message-ID: Assuming what you want is a ZIP with WildFly and client adapter then yes, but not sure it works. What's this for? On 6 November 2015 at 13:12, Pedro Igor Silva wrote: > Hi, > > Is keycloak-adapter-feature-pack the right pack to use to build > distributions to deploy client applications only, which require only the > adapters ? In this case, WFLY ones ... > > Regards. > Pedro Igor > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/41ff0859/attachment.html From psilva at redhat.com Fri Nov 6 08:06:10 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Fri, 6 Nov 2015 08:06:10 -0500 (EST) Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: References: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> Message-ID: <1605280911.5773222.1446815170611.JavaMail.zimbra@redhat.com> Would like to provide a ready-to-use WFLY distribution for clients with the necessary stuff to integrate with the AuthZ server. I will also provide the "overlay" stuff for those with an existing WFLY installation. This is related with [1]. The server is pretty much done, I've created a feature pack on top of KC server-feature-pack. Now I would like to distribute something for client apps only. Any thoughts ? [1] https://github.com/pedroigor/keycloak-authz/issues/21 ----- Original Message ----- From: "Stian Thorgersen" To: "Pedro Igor Silva" Cc: "keycloak dev" Sent: Friday, November 6, 2015 10:53:45 AM Subject: Re: [keycloak-dev] Adapter Feature Pack Assuming what you want is a ZIP with WildFly and client adapter then yes, but not sure it works. What's this for? On 6 November 2015 at 13:12, Pedro Igor Silva wrote: > Hi, > > Is keycloak-adapter-feature-pack the right pack to use to build > distributions to deploy client applications only, which require only the > adapters ? In this case, WFLY ones ... > > Regards. > Pedro Igor > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From sthorger at redhat.com Fri Nov 6 08:14:46 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 14:14:46 +0100 Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: <1605280911.5773222.1446815170611.JavaMail.zimbra@redhat.com> References: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> <1605280911.5773222.1446815170611.JavaMail.zimbra@redhat.com> Message-ID: For clients you should provide an overlay zip that can be installed into existing WildFly. That's what we do for Keycloak. Reasoning behind this is simple, users want to add adapter to the version of WildFly they are using. They may have configuration changes or deployments added. They may also have modules or even other extensions loaded. Ideally there should be a mechanism provided by WildFly that enables installing a feature pack into an existing WildFly, but there isn't. That's why we have a ZIP + a installation CLI script instead. Take a look at http://keycloak.github.io/docs/userguide/keycloak-server/html/ch08.html#jboss-adapter . On 6 November 2015 at 14:06, Pedro Igor Silva wrote: > Would like to provide a ready-to-use WFLY distribution for clients with > the necessary stuff to integrate with the AuthZ server. I will also provide > the "overlay" stuff for those with an existing WFLY installation. > > This is related with [1]. The server is pretty much done, I've created a > feature pack on top of KC server-feature-pack. Now I would like to > distribute something for client apps only. Any thoughts ? > > [1] https://github.com/pedroigor/keycloak-authz/issues/21 > > ----- Original Message ----- > From: "Stian Thorgersen" > To: "Pedro Igor Silva" > Cc: "keycloak dev" > Sent: Friday, November 6, 2015 10:53:45 AM > Subject: Re: [keycloak-dev] Adapter Feature Pack > > Assuming what you want is a ZIP with WildFly and client adapter then yes, > but not sure it works. > > What's this for? > > On 6 November 2015 at 13:12, Pedro Igor Silva wrote: > > > Hi, > > > > Is keycloak-adapter-feature-pack the right pack to use to build > > distributions to deploy client applications only, which require only the > > adapters ? In this case, WFLY ones ... > > > > Regards. > > Pedro Igor > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/456fe47d/attachment.html From ssilvert at redhat.com Fri Nov 6 08:27:03 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Fri, 06 Nov 2015 08:27:03 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: References: <5639253A.7090408@redhat.com> <563B611F.2070304@redhat.com> <563BAD00.5050609@redhat.com> <563BBECA.8090509@redhat.com> <563C9DA9.7090704@redhat.com> Message-ID: <563CAAA7.20305@redhat.com> On 11/6/2015 7:52 AM, Stian Thorgersen wrote: > I disagree. Firstly we're not adding Permissions now. Secondly the > Group is either part of the Subject or it's the equivalent of a Role, > that depends on policies (which are SE), not the mapping of > Permissions to the Subject. We have a completely different idea of what a Group is. That's OK. At least we are finally using the same vocabulary. For the sake of this discussion, does it matter that we're not adding Permissions now? We are planning this, right? There are several implementations of NIST RBAC out there. I wonder if there is a good open source one we could just plug in. > > Following your argument if you had allowed assigning permissions based > on dob then the dob of a user would be an SE. > > On 6 November 2015 at 13:31, Stan Silvert > wrote: > > On 11/6/2015 1:48 AM, Stian Thorgersen wrote: >> >> >> On 5 November 2015 at 21:40, Stan Silvert > > wrote: >> >> On 11/5/2015 2:36 PM, Stian Thorgersen wrote: >>> We're only providing parts of RBAC now. The complete picture >>> is what Pedro is working on with his AuthZ service. >> Yea, as I understand it, Pedro is doing P. (P for Pedro!) >> And also, he's filling in the rest of the gaps surrounding P. >>> >>> From the definitions above we're actually only providing S >>> and R. SE is not a group as a group doesn't provide any >>> permissions. >> Maybe that's a good reason to stick with the definitions >> below. I see "Group" as a way to implement the mapping >> called for in SE. But it doesn't have to be that way. >> >> >> Group is not SE, I'm pretty sure of that. A group is just an >> "attribute" of the subject. It doesn't "provide" any permissions >> or any mapping between subjects and permissions. > Let's say you allow Permissions and Roles to be assigned to a > Group. And you also design it such that any Subject who becomes a > member of the group also gets the assigned Permissions and > Roles. In that case, you have implemented a Group that acts as > the SE. > >> >>> >>> I'm not 100% sure what the group would be in the above, but >>> I would think it's just part of S. A group is simply a means >>> of assigning a role to a group of users. >>> >>> >>> On 5 November 2015 at 20:24, Stan Silvert >>> > wrote: >>> >>> We could do a lot worse than just following the basic >>> RBAC design described on Wikipedia: >>> https://en.wikipedia.org/wiki/Role-based_access_control >>> >>> Right now we're arguing over both definitions AND >>> implementations. It's impossible to design this over >>> email if we can't even settle on definitions. >>> >>> Therefore, I propose we just use the definitions in >>> wikipedia. At least it's neutral. >>> >>> * S = Subject = A person or automated agent >>> * R = Role = Job function or title which defines an >>> authority level >>> * P = Permissions = An approval of a mode of access to >>> a resource >>> * SE = Session = A mapping involving S, R and/or P >>> * SA = Subject Assignment >>> * PA = Permission Assignment >>> * RH = Partially ordered Role Hierarchy. RH can also >>> be written: ? (The notation: x ? y means that x >>> inherits the permissions of y.) >>> o A subject can have multiple roles. >>> o A role can have multiple subjects. >>> o A role can have many permissions. >>> o A permission can be assigned to many roles. >>> o An operation can be assigned many permissions. >>> o A permission can be assigned to many operations. >>> >>> >>> Note: In my mind, S = keycloak user, and SE = keycloak >>> group. But whatever, as long as we agree on definitions >>> we can then decide what flavor of RBAC to implement. >>> >>> >>> On 11/5/2015 1:44 PM, Stian Thorgersen wrote: >>>> >>>> >>>> On 5 November 2015 at 15:01, Bill Burke >>>> > wrote: >>>> >>>> >>>> >>>> On 11/5/2015 6:23 AM, Stian Thorgersen wrote: >>>> >>>> >>>> >>>> On 3 November 2015 at 22:20, Bill Burke >>>> >>>> >>> >> wrote: >>>> >>>> In my previous email I talked about >>>> combining Groups and Role >>>> Namespaces. Now I want to talk about User >>>> Groups vs. Client Groups. >>>> >>>> User Groups would manage a set of users. >>>> Members would automatically >>>> inherit a set of "permissions": a set of >>>> roles. User Groups would also >>>> provide a set of attributes that the user >>>> inherits. >>>> >>>> >>>> Permission != role >>>> >>>> >>>> I'd like to introduce the concept of a >>>> Client Group. Client Group would >>>> have: >>>> >>>> * Roles - basically a role namespace >>>> >>>> >>>> -1 Having roles tied to a client or client >>>> group is exactly what we >>>> should go away from. IMO role namespaces should >>>> be a completely separate >>>> thing. >>>> >>>> >>>> I don't agree at all. If User Groups and Client >>>> Groups exist, there is no need for role namespaces. >>>> It is stupid to have to create another concept >>>> (role namespace) to define the roles one specific >>>> client or a group of clients expects. >>>> >>>> >>>> I've never the concept of realm and client roles. It's >>>> been difficult to explain and strange to use. I've >>>> always just used realm roles. It's a strange and >>>> limiting concept. Introducing client groups with >>>> further places to define roles just makes matters even >>>> worse. Now users have two go 3 different places to >>>> define roles: >>>> >>>> * Realm >>>> * Client Groups >>>> * Clients >>>> >>>> What happens if a client group and a client both have >>>> the same role by the way? >>>> >>>> It's a strange limitation. At least personally if I was >>>> using Keycloak I would simply use realm roles alone and >>>> define my own hierarchy with a delimiter. >>>> >>>> It's much better to have a single place to define >>>> roles, under the roles tab. Then allow users can define >>>> the namespaces/hierarchy they want. >>>> >>>> Role namespaces are easier to deal with and at the same >>>> time more flexible. >>>> >>>> I just don't see any reason why we would have roles >>>> specific to a client or client group. >>>> >>>> >>>> If you combine Role namespace and Groups you can >>>> define things like a group admin role. Roles that >>>> mean something to the group. >>>> >>>> >>>> Each Client Group would have some default >>>> roles defined. i.e. roles >>>> that allow a user to edit any client in the >>>> client group. >>>> >>>> >>>> I don't understand this >>>> >>>> >>>> A Client Group could have a "client group admin" >>>> role. If a user has that role it can manage clients >>>> in the group. Another role might be "client >>>> membership admin". This role allows a user to add >>>> or remove clients from the group. >>>> >>>> Conversely, user groups could have a "user group >>>> admin". When granted, this role allows a user to >>>> manage users in the group. YOu can also do things >>>> like define a "Manager" role for the group. This >>>> "Manager" would be granted "user group admin" >>>> privileges and also granted access to other systems >>>> like "HR", "Attendence", "Benefits", etc. >>>> >>>> I think this permission concept should be built >>>> into Keycloak as it is a core feature. I'll write >>>> u a separate email about this. >>>> >>>> >>>> This is another reason why we need role namespaces. >>>> With a role namespace we can define internal roles that >>>> then don't end up conflicting with users own roles. For >>>> example as we have a role admin atm users can't define >>>> their own admin role and will have to name it differently. >>>> >>>> I think the fact that we have internal abstract clients >>>> to be able to create a namespace for internal admin >>>> roles speaks for itself. >>>> >>>> >>>> >>>> >>>> >>>> Each Client would have the same >>>> configuration options. They would be >>>> able to have an additional set of roles, >>>> permissions, scope, and >>>> overridable Protocol Policies. >>>> >>>> >>>> Same comment as above - why would a client have >>>> roles/permissions? I >>>> assume we where moving away from that with role >>>> namespaces >>>> >>>> >>>> Again, I think role namespaces are redundant. >>>> >>>> A client can define a set of roles that it offers. >>>> A service account (the client) can have roles >>>> associated with it so it can do certain actions. >>>> >>>> >>>> Some will want to have roles associated with a client >>>> (email-user), but others have organizational wide roles >>>> (manager, sales-guy, etc..). Role namespaces can deal >>>> with both, but client roles can't. >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Bill Burke >>>> JBoss, a division of Red Hat >>>> http://bill.burkecentral.com >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/20507134/attachment-0001.html From erik.mulder at docdatapayments.com Fri Nov 6 08:29:00 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Fri, 6 Nov 2015 14:29:00 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> Thanks for pointing me explicitly to the SPI documentation. Of course that is exactly what I was looking for in my original question. I don't know how I overlooked this earlier! Probably I was not picking it up, because of almost a decade of developing on Spring projects, where this type of thing works differently. :-) I tried a quick test with a jar with one extra ProtocolMapper configured, put it in the providers folder and it worked like a charm! As for the JPA: We'll probably go with your suggestion of the separate EntityManagerFactory. Indeed there seems to be no way to 'programmatically extend' the list of entity classes in JPA besides editing or overwriting the persistence.xml. As you probably know it can be done in Hibernate, but I guess KeyCloak wants to stick to a generic JPA solution. That said, we might consider the Hibernate specific solution for our case, since being able to switch the JPA provider is not a requirement for us. And keeping the same connection/transaction is a lot easier in reasoning and debugging. On 05/11/15 10:52, Stian Thorgersen wrote: The way to extend Keycloak is by implementing your own custom providers of the many SPIs we provide. Some SPIs are more stable (so marked as public) and others are not (so marked as private). If there are things that you want to customize that can't be done with an existing SPI then let us know and we may consider adding additional SPIs. On 4 November 2015 at 17:16, Erik Mulder > wrote: Thanks for your response! Indeed we already did a proof of concept where we added a custom mapper the way you described (didn't know it was 'protected' territory :). The question is: do we have to override the file 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper in the original project or is there another way where we don't need to touch the original sources and keep all our changes in a separate project? And how can we do it such that it stays easy to upgrade to newer KeyCloak releases? Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at the docs (http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) and examples for other provider (https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory) As for JPA: it would be easier to integrate with the existing JPA project. Again we are wondering whether to start modifying original sources (like persistence.xml) or try to 'externalize' our changes somehow and integrate them using existing 'hooks' in the system or maybe merge projects during build. Maybe there is no good answer to this and we'll always be having some manual merge pains when upgrading to new KeyCloak versions. We just wanted to check if there are preferred ways to add functionality with the least amount of impact on the original sources. I initially wanted the ability to add custom entities to the JpaConnectProvider, but couldn't find a way to define entities programatically with JPA. To add your own persistence.xml you would have to define your own implementation of JpaConnectionProvider and change what is loaded by default (connectionsJpa/provider attribute in keycloak-server.json). Alternative, which is cleaner, but you end up with separate connection/transaction, is to create your own EntityManagerFactory. If it's only used by one provider (for example a custom UserFederationProvider) there's no need to add a connect provider (that's just a way to share one EntityManagerFactory between multiple providers) and you can just create it in the MyUserFederationProviderFactory. On 04/11/15 15:30, Bill Burke wrote: > Custom mappers should be possible. I didn't document it as I wasn't > sure if we wanted to make the SPI public. Custom mappers should just > follow the Provider SPI and they will be picked up. If you see the > META-INF/services/... file in the resources directory of the "services" > or "broker" modules you'll see how to set this up. > > As for extending the JPA datamodel, what you could do is write a new JPA > Connections Provider and plug that in. See connections/jpa. I'm not > sure how you would handle the liquibase db migration. > > On 11/4/2015 6:03 AM, Erik Mulder wrote: >> Hi everybody, >> >> Quick intro: I?m part of a development team in The Netherlands that is >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >> this and will use OpenID Connect to secure our REST services. It?s a >> great product and seems to be the only one having both support for all >> kinds of security standards and a model and GUI for users and roles. >> Thanks for creating it! J >> >> (if this should be asked instead on the users mailing list, please >> correct me and I?ll post it there) >> >> So far, so good, but we have some extra requirements that do not fit >> into the base KeyCloak data model. See below for details if you?re >> interested. My question is: what is the preferred way / best practice to >> extend the functionality of KeyCloak while keeping the impact on the >> original sources to a minimum? Of course we could just fork the most >> recent version and start hacking away, but we?d like to be able to >> upgrade to newer versions of KeyCloak without too much hassle. >> Possibilities that we?ve come up with so far: >> >> 1.Create completely separate modules that will extend the functionality >> the way we need. >> >> 2.Fork on Github, apply custom changes, and try to merge in updates from >> the master / release branches / tags >> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >> as Truezip >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >> manipulate zip files by adding/removing/replacing or Shade >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple >> jars to 1 'uber-jar' containing the contents of both and when >> overlapping decide on conflicts through configuration. >> >> Of course number 1 is preferred, but I do not see how to add custom >> mappers or JPA entities without making changes in the original module >> files. The other options seem like valid alternatives, but maybe there >> is better / standard way to do this. So any help / insight / shared >> experience on this is much appreciated, thanks! >> >> Kind regards, >> >> Erik Mulder >> >> Senior Software Engineer >> >> Docdata Payments ? NL >> >> P.S. Details on why we want to extend the KeyCloak data model: (any >> feedback on the contents of this P.S. is also welcome!) >> >> Our clients are merchants that have several webshops. We manage their >> online payments (shopping cart checkout). We want to be able to let a >> merchant manage their own users and let a user have different roles for >> different webshops within the same merchant. The overall possible roles >> are fixed though, no specific roles per merchant. We could create a >> separate realm for every merchant, but then we need to duplicate all >> roles every time. Furthermore, in KeyCloak there is no concept of a role >> within a certain context. This is very understandable, since every >> situation has it?s own requirements. We did a proof of concept by adding >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >> and adding a custom mapper that can put this information on the Access >> token. Worked like a charm! But it does need some changes in the >> KeyCloak modules and sources to work, hence the question above. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/296bf4cb/attachment.html From psilva at redhat.com Fri Nov 6 08:31:22 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Fri, 6 Nov 2015 08:31:22 -0500 (EST) Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: References: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> <1605280911.5773222.1446815170611.JavaMail.zimbra@redhat.com> Message-ID: <2028970915.5792920.1446816682776.JavaMail.zimbra@redhat.com> Yeah, I saw that. But I was thinking that provide a ready-to-use distribution would be useful as well, just for evaluation and testing purposes. So you just unzip, start the server and deploy apps. That would make things more easier for people evaluating the AuthZ Server. I'm also considering overlay. I've just tested the adapter-feature-pack and yes, it seems it is not working properly. Think I'm going to forget about it and just do overlay. Btw, aren't Keycloak WFLY adapters based on WFLY 8/9 only ? ----- Original Message ----- From: "Stian Thorgersen" To: "Pedro Igor Silva" Cc: "keycloak dev" Sent: Friday, November 6, 2015 11:14:46 AM Subject: Re: [keycloak-dev] Adapter Feature Pack For clients you should provide an overlay zip that can be installed into existing WildFly. That's what we do for Keycloak. Reasoning behind this is simple, users want to add adapter to the version of WildFly they are using. They may have configuration changes or deployments added. They may also have modules or even other extensions loaded. Ideally there should be a mechanism provided by WildFly that enables installing a feature pack into an existing WildFly, but there isn't. That's why we have a ZIP + a installation CLI script instead. Take a look at http://keycloak.github.io/docs/userguide/keycloak-server/html/ch08.html#jboss-adapter . On 6 November 2015 at 14:06, Pedro Igor Silva wrote: > Would like to provide a ready-to-use WFLY distribution for clients with > the necessary stuff to integrate with the AuthZ server. I will also provide > the "overlay" stuff for those with an existing WFLY installation. > > This is related with [1]. The server is pretty much done, I've created a > feature pack on top of KC server-feature-pack. Now I would like to > distribute something for client apps only. Any thoughts ? > > [1] https://github.com/pedroigor/keycloak-authz/issues/21 > > ----- Original Message ----- > From: "Stian Thorgersen" > To: "Pedro Igor Silva" > Cc: "keycloak dev" > Sent: Friday, November 6, 2015 10:53:45 AM > Subject: Re: [keycloak-dev] Adapter Feature Pack > > Assuming what you want is a ZIP with WildFly and client adapter then yes, > but not sure it works. > > What's this for? > > On 6 November 2015 at 13:12, Pedro Igor Silva wrote: > > > Hi, > > > > Is keycloak-adapter-feature-pack the right pack to use to build > > distributions to deploy client applications only, which require only the > > adapters ? In this case, WFLY ones ... > > > > Regards. > > Pedro Igor > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > From erik.mulder at docdatapayments.com Fri Nov 6 08:32:46 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Fri, 6 Nov 2015 14:32:46 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <469783768.4549663.1446717984477.JavaMail.zimbra@redhat.com> Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B068@DDPEX01.DDP.dcloud.local> I guess you are referring to this branch: https://github.com/pedroigor/keycloak/tree/keycloak-authz-modified Thanks, that might be useful to us. I'm somewhat new to the whole SPI approach (I've only done Spring projects so far), so I'm not quite sure what kind of functionality / extension hook your branch provides exactly. But we'll figure it out when we get more accustomed to the SPI API. If I have any questions left when we get to the point of integrating this, I know where to find you. :-) On 05/11/15 11:06, Pedro Igor Silva wrote: > Btw, I've already worked on a SPI to plug custom endpoints to both Admin and Realm API. I'm still holding a PR with these changes, but if you need it I can send them. Just to avoid re-work :) > > ----- Original Message ----- > From: "Stian Thorgersen" > To: "Erik Mulder" > Cc: keycloak-dev at lists.jboss.org > Sent: Thursday, November 5, 2015 7:52:13 AM > Subject: Re: [keycloak-dev] Preferred way to make KeyCloak custom changes > > The way to extend Keycloak is by implementing your own custom providers of the many SPIs we provide. Some SPIs are more stable (so marked as public) and others are not (so marked as private). If there are things that you want to customize that can't be done with an existing SPI then let us know and we may consider adding additional SPIs. > > On 4 November 2015 at 17:16, Erik Mulder < erik.mulder at docdatapayments.com > wrote: > > > Thanks for your response! > > Indeed we already did a proof of concept where we added a custom mapper > the way you described (didn't know it was 'protected' territory :). The > question is: do we have to override the file > 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper > in the original project or is there another way where we don't need to > touch the original sources and keep all our changes in a separate > project? And how can we do it such that it stays easy to upgrade to > newer KeyCloak releases? > > Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at the docs ( http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html ) and examples for other provider ( https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory ) > > > > As for JPA: it would be easier to integrate with the existing JPA > project. Again we are wondering whether to start modifying original > sources (like persistence.xml) or try to 'externalize' our changes > somehow and integrate them using existing 'hooks' in the system or maybe > merge projects during build. > > Maybe there is no good answer to this and we'll always be having some > manual merge pains when upgrading to new KeyCloak versions. We just > wanted to check if there are preferred ways to add functionality with > the least amount of impact on the original sources. > > I initially wanted the ability to add custom entities to the JpaConnectProvider, but couldn't find a way to define entities programatically with JPA. To add your own persistence.xml you would have to define your own implementation of JpaConnectionProvider and change what is loaded by default (connectionsJpa/provider attribute in keycloak-server.json). > > Alternative, which is cleaner, but you end up with separate connection/transaction, is to create your own EntityManagerFactory. If it's only used by one provider (for example a custom UserFederationProvider) there's no need to add a connect provider (that's just a way to share one EntityManagerFactory between multiple providers) and you can just create it in the MyUserFederationProviderFactory. > > > > > > On 04/11/15 15:30, Bill Burke wrote: >> Custom mappers should be possible. I didn't document it as I wasn't >> sure if we wanted to make the SPI public. Custom mappers should just >> follow the Provider SPI and they will be picked up. If you see the >> META-INF/services/... file in the resources directory of the "services" >> or "broker" modules you'll see how to set this up. >> >> As for extending the JPA datamodel, what you could do is write a new JPA >> Connections Provider and plug that in. See connections/jpa. I'm not >> sure how you would handle the liquibase db migration. >> >> On 11/4/2015 6:03 AM, Erik Mulder wrote: >>> Hi everybody, >>> >>> Quick intro: I?m part of a development team in The Netherlands that is >>> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >>> this and will use OpenID Connect to secure our REST services. It?s a >>> great product and seems to be the only one having both support for all >>> kinds of security standards and a model and GUI for users and roles. >>> Thanks for creating it! J >>> >>> (if this should be asked instead on the users mailing list, please >>> correct me and I?ll post it there) >>> >>> So far, so good, but we have some extra requirements that do not fit >>> into the base KeyCloak data model. See below for details if you?re >>> interested. My question is: what is the preferred way / best practice to >>> extend the functionality of KeyCloak while keeping the impact on the >>> original sources to a minimum? Of course we could just fork the most >>> recent version and start hacking away, but we?d like to be able to >>> upgrade to newer versions of KeyCloak without too much hassle. >>> Possibilities that we?ve come up with so far: >>> >>> 1.Create completely separate modules that will extend the functionality >>> the way we need. >>> >>> 2.Fork on Github, apply custom changes, and try to merge in updates from >>> the master / release branches / tags >>> >>> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >>> as Truezip >>> ( http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html ) - >>> manipulate zip files by adding/removing/replacing or Shade >>> ( http://maven.apache.org/plugins/maven-shade-plugin/ ) - combine multiple >>> jars to 1 'uber-jar' containing the contents of both and when >>> overlapping decide on conflicts through configuration. >>> >>> Of course number 1 is preferred, but I do not see how to add custom >>> mappers or JPA entities without making changes in the original module >>> files. The other options seem like valid alternatives, but maybe there >>> is better / standard way to do this. So any help / insight / shared >>> experience on this is much appreciated, thanks! >>> >>> Kind regards, >>> >>> Erik Mulder >>> >>> Senior Software Engineer >>> >>> Docdata Payments ? NL >>> >>> P.S. Details on why we want to extend the KeyCloak data model: (any >>> feedback on the contents of this P.S. is also welcome!) >>> >>> Our clients are merchants that have several webshops. We manage their >>> online payments (shopping cart checkout). We want to be able to let a >>> merchant manage their own users and let a user have different roles for >>> different webshops within the same merchant. The overall possible roles >>> are fixed though, no specific roles per merchant. We could create a >>> separate realm for every merchant, but then we need to duplicate all >>> roles every time. Furthermore, in KeyCloak there is no concept of a role >>> within a certain context. This is very understandable, since every >>> situation has it?s own requirements. We did a proof of concept by adding >>> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >>> and adding a custom mapper that can put this information on the Access >>> token. Worked like a charm! But it does need some changes in the >>> KeyCloak modules and sources to work, hence the question above. >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > From mposolda at redhat.com Fri Nov 6 08:36:36 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 6 Nov 2015 14:36:36 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: <563B7FBF.8060701@redhat.com> References: <563B3371.8040402@redhat.com> <563B7FBF.8060701@redhat.com> Message-ID: <563CACE4.7050909@redhat.com> On 05/11/15 17:11, Bill Burke wrote: > > On 11/5/2015 5:46 AM, Marek Posolda wrote: >> I wonder if we should improve handling of custom attributes in >> freemarker templates and their validation. Let's assume that I want to >> add custom attribute "birthday" and I want to add it to all screens >> where user can create/edit account. I can see those issues: >> >> - Admin will need to edit 3 separate freemarker templates (registration, >> account management, update profile page) to have this attribute >> displayed on all those places. >> > we've discussed this before. The problem is formatting in each of the > UIs. Often ordering of attributes is important. Grouping of attributes > is a must too. > > i.e. Address, State, Country, Postal Code > > These 4 attribute must be grouped together, and address must come before > state, and so on. > > You might also have a "Billing Address" group that needs to come after > Home Address group. I am not seeing an issue with ordering? The same order you configure in admin console, the same is used in template. It's classic sorted list, not special tricks needed IMO. The grouping some attributes can be easily addressed too IMO. > > So, we'd have an automatic way for displaying attributes, then the > developer would think "that looks like shit" and want to format things > himself. Stian seems to think that CSS will solve this problem, but I'm > not convinced. I agree we can't address all possible sort of issues, but IMO it's good to have something, which will address 90% of cases? We can provide types (like combobox "Gender" with values "male" / "female" ). We can also handle multivalued attributes. And most importantly we can provide validations for all 3 screens. People won't need to write their own Java validators when they want something simple like ensure "mandatory" field present or validate with regex. They will still have possibility to inject the custom validator if they want it. At least validation is must IMO as there is no way to add custom validation for account mgmt or update profile right now. What we can't easily do is client-side validation and stuff for display/hide something based on value of other attributes (For example display attribute "Favourite car sign" just if selected gender is "male" ). But not sure if this is often requirement. I am seeing lot of similarity with your kc-provider-config directive in admin console. It can't address all "generic" sort of things and client-side stuff etc. But for most cases, it's sufficient. > > Also the look and feel could be quite different between registration, > update profile, account management, and the admin profile. IMO, it > would end up being easier to just edit the freemarker pages directly > than to have to define each attribute and how it is grouped, ordered, > and displayed on each of the pages within an admin console UI. > > There could be 3 checkboxes where can admin select if he wants to add field on registration, account mgmt and update profile. I am not sure if we need anything for admin console as we already have "Attributes" tab for users, which is almost ok IMO (we don't have nice solution for multivalued attributes). Marek From ssilvert at redhat.com Fri Nov 6 08:37:11 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Fri, 06 Nov 2015 08:37:11 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: References: <5639253A.7090408@redhat.com> <563B611F.2070304@redhat.com> <563BAD00.5050609@redhat.com> <563BBECA.8090509@redhat.com> <563C9DA9.7090704@redhat.com> Message-ID: <563CAD07.1010408@redhat.com> On 11/6/2015 7:52 AM, Stian Thorgersen wrote: > I disagree. Firstly we're not adding Permissions now. Secondly the > Group is either part of the Subject or it's the equivalent of a Role, > that depends on policies (which are SE), not the mapping of > Permissions to the Subject. > > Following your argument if you had allowed assigning permissions based > on dob then the dob of a user would be an SE. RBAC considers the traditional Group to be a "loophole". The session is something that takes on some of the responsibilities I was thinking of for groups. So in this scheme, I guess you don't have Groups. Part of a Group's responsibility is handled by the role and part is handled by the session.: How are roles different from groups? There is a superficial similarity between RBAC roles and traditional groups. As normally implemented, a group is a collection of users, rather than a collection of permissions, and permissions can be associated with both users and the groups to which they belong. The ability to tie permissions directly to users in a group-based mechanism can be regarded as a "loophole" that makes it difficult to control user-permission relationships. RBAC requires all access through roles, and permissions are connected only to roles, not directly to users. Another aspect of RBAC that distinguishes it from traditional group mechanisms is the concept of a session, which allows activation of a subset of roles assigned to a user. Core RBAC includes those systems with a robust group/ACL mechanism that supports the construction of a many-to-many relation among users and permissions. http://csrc.nist.gov/groups/SNS/rbac/faq.html > > On 6 November 2015 at 13:31, Stan Silvert > wrote: > > On 11/6/2015 1:48 AM, Stian Thorgersen wrote: >> >> >> On 5 November 2015 at 21:40, Stan Silvert > > wrote: >> >> On 11/5/2015 2:36 PM, Stian Thorgersen wrote: >>> We're only providing parts of RBAC now. The complete picture >>> is what Pedro is working on with his AuthZ service. >> Yea, as I understand it, Pedro is doing P. (P for Pedro!) >> And also, he's filling in the rest of the gaps surrounding P. >>> >>> From the definitions above we're actually only providing S >>> and R. SE is not a group as a group doesn't provide any >>> permissions. >> Maybe that's a good reason to stick with the definitions >> below. I see "Group" as a way to implement the mapping >> called for in SE. But it doesn't have to be that way. >> >> >> Group is not SE, I'm pretty sure of that. A group is just an >> "attribute" of the subject. It doesn't "provide" any permissions >> or any mapping between subjects and permissions. > Let's say you allow Permissions and Roles to be assigned to a > Group. And you also design it such that any Subject who becomes a > member of the group also gets the assigned Permissions and > Roles. In that case, you have implemented a Group that acts as > the SE. > >> >>> >>> I'm not 100% sure what the group would be in the above, but >>> I would think it's just part of S. A group is simply a means >>> of assigning a role to a group of users. >>> >>> >>> On 5 November 2015 at 20:24, Stan Silvert >>> > wrote: >>> >>> We could do a lot worse than just following the basic >>> RBAC design described on Wikipedia: >>> https://en.wikipedia.org/wiki/Role-based_access_control >>> >>> Right now we're arguing over both definitions AND >>> implementations. It's impossible to design this over >>> email if we can't even settle on definitions. >>> >>> Therefore, I propose we just use the definitions in >>> wikipedia. At least it's neutral. >>> >>> * S = Subject = A person or automated agent >>> * R = Role = Job function or title which defines an >>> authority level >>> * P = Permissions = An approval of a mode of access to >>> a resource >>> * SE = Session = A mapping involving S, R and/or P >>> * SA = Subject Assignment >>> * PA = Permission Assignment >>> * RH = Partially ordered Role Hierarchy. RH can also >>> be written: ? (The notation: x ? y means that x >>> inherits the permissions of y.) >>> o A subject can have multiple roles. >>> o A role can have multiple subjects. >>> o A role can have many permissions. >>> o A permission can be assigned to many roles. >>> o An operation can be assigned many permissions. >>> o A permission can be assigned to many operations. >>> >>> >>> Note: In my mind, S = keycloak user, and SE = keycloak >>> group. But whatever, as long as we agree on definitions >>> we can then decide what flavor of RBAC to implement. >>> >>> >>> On 11/5/2015 1:44 PM, Stian Thorgersen wrote: >>>> >>>> >>>> On 5 November 2015 at 15:01, Bill Burke >>>> > wrote: >>>> >>>> >>>> >>>> On 11/5/2015 6:23 AM, Stian Thorgersen wrote: >>>> >>>> >>>> >>>> On 3 November 2015 at 22:20, Bill Burke >>>> >>>> >>> >> wrote: >>>> >>>> In my previous email I talked about >>>> combining Groups and Role >>>> Namespaces. Now I want to talk about User >>>> Groups vs. Client Groups. >>>> >>>> User Groups would manage a set of users. >>>> Members would automatically >>>> inherit a set of "permissions": a set of >>>> roles. User Groups would also >>>> provide a set of attributes that the user >>>> inherits. >>>> >>>> >>>> Permission != role >>>> >>>> >>>> I'd like to introduce the concept of a >>>> Client Group. Client Group would >>>> have: >>>> >>>> * Roles - basically a role namespace >>>> >>>> >>>> -1 Having roles tied to a client or client >>>> group is exactly what we >>>> should go away from. IMO role namespaces should >>>> be a completely separate >>>> thing. >>>> >>>> >>>> I don't agree at all. If User Groups and Client >>>> Groups exist, there is no need for role namespaces. >>>> It is stupid to have to create another concept >>>> (role namespace) to define the roles one specific >>>> client or a group of clients expects. >>>> >>>> >>>> I've never the concept of realm and client roles. It's >>>> been difficult to explain and strange to use. I've >>>> always just used realm roles. It's a strange and >>>> limiting concept. Introducing client groups with >>>> further places to define roles just makes matters even >>>> worse. Now users have two go 3 different places to >>>> define roles: >>>> >>>> * Realm >>>> * Client Groups >>>> * Clients >>>> >>>> What happens if a client group and a client both have >>>> the same role by the way? >>>> >>>> It's a strange limitation. At least personally if I was >>>> using Keycloak I would simply use realm roles alone and >>>> define my own hierarchy with a delimiter. >>>> >>>> It's much better to have a single place to define >>>> roles, under the roles tab. Then allow users can define >>>> the namespaces/hierarchy they want. >>>> >>>> Role namespaces are easier to deal with and at the same >>>> time more flexible. >>>> >>>> I just don't see any reason why we would have roles >>>> specific to a client or client group. >>>> >>>> >>>> If you combine Role namespace and Groups you can >>>> define things like a group admin role. Roles that >>>> mean something to the group. >>>> >>>> >>>> Each Client Group would have some default >>>> roles defined. i.e. roles >>>> that allow a user to edit any client in the >>>> client group. >>>> >>>> >>>> I don't understand this >>>> >>>> >>>> A Client Group could have a "client group admin" >>>> role. If a user has that role it can manage clients >>>> in the group. Another role might be "client >>>> membership admin". This role allows a user to add >>>> or remove clients from the group. >>>> >>>> Conversely, user groups could have a "user group >>>> admin". When granted, this role allows a user to >>>> manage users in the group. YOu can also do things >>>> like define a "Manager" role for the group. This >>>> "Manager" would be granted "user group admin" >>>> privileges and also granted access to other systems >>>> like "HR", "Attendence", "Benefits", etc. >>>> >>>> I think this permission concept should be built >>>> into Keycloak as it is a core feature. I'll write >>>> u a separate email about this. >>>> >>>> >>>> This is another reason why we need role namespaces. >>>> With a role namespace we can define internal roles that >>>> then don't end up conflicting with users own roles. For >>>> example as we have a role admin atm users can't define >>>> their own admin role and will have to name it differently. >>>> >>>> I think the fact that we have internal abstract clients >>>> to be able to create a namespace for internal admin >>>> roles speaks for itself. >>>> >>>> >>>> >>>> >>>> >>>> Each Client would have the same >>>> configuration options. They would be >>>> able to have an additional set of roles, >>>> permissions, scope, and >>>> overridable Protocol Policies. >>>> >>>> >>>> Same comment as above - why would a client have >>>> roles/permissions? I >>>> assume we where moving away from that with role >>>> namespaces >>>> >>>> >>>> Again, I think role namespaces are redundant. >>>> >>>> A client can define a set of roles that it offers. >>>> A service account (the client) can have roles >>>> associated with it so it can do certain actions. >>>> >>>> >>>> Some will want to have roles associated with a client >>>> (email-user), but others have organizational wide roles >>>> (manager, sales-guy, etc..). Role namespaces can deal >>>> with both, but client roles can't. >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Bill Burke >>>> JBoss, a division of Red Hat >>>> http://bill.burkecentral.com >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/bb4da7a3/attachment-0001.html From sthorger at redhat.com Fri Nov 6 08:41:45 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 14:41:45 +0100 Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: <2028970915.5792920.1446816682776.JavaMail.zimbra@redhat.com> References: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> <1605280911.5773222.1446815170611.JavaMail.zimbra@redhat.com> <2028970915.5792920.1446816682776.JavaMail.zimbra@redhat.com> Message-ID: On 6 November 2015 at 14:31, Pedro Igor Silva wrote: > Yeah, I saw that. But I was thinking that provide a ready-to-use > distribution would be useful as well, just for evaluation and testing > purposes. So you just unzip, start the server and deploy apps. That would > make things more easier for people evaluating the AuthZ Server. > I'm also considering overlay. I've just tested the adapter-feature-pack > and yes, it seems it is not working properly. Think I'm going to forget > about it and just do overlay. > If you want a nice simple way for folks to try it out create a demo dist - we have our demo dist which includes wildfly + server + adapter + examples. Single download for everything. We just use the overlay to create it. > > Btw, aren't Keycloak WFLY adapters based on WFLY 8/9 only ? > WFL9 works on WFL10 AFAIK > > ----- Original Message ----- > From: "Stian Thorgersen" > To: "Pedro Igor Silva" > Cc: "keycloak dev" > Sent: Friday, November 6, 2015 11:14:46 AM > Subject: Re: [keycloak-dev] Adapter Feature Pack > > For clients you should provide an overlay zip that can be installed into > existing WildFly. That's what we do for Keycloak. Reasoning behind this is > simple, users want to add adapter to the version of WildFly they are using. > They may have configuration changes or deployments added. They may also > have modules or even other extensions loaded. > > Ideally there should be a mechanism provided by WildFly that enables > installing a feature pack into an existing WildFly, but there isn't. That's > why we have a ZIP + a installation CLI script instead. Take a look at > > http://keycloak.github.io/docs/userguide/keycloak-server/html/ch08.html#jboss-adapter > . > > On 6 November 2015 at 14:06, Pedro Igor Silva wrote: > > > Would like to provide a ready-to-use WFLY distribution for clients with > > the necessary stuff to integrate with the AuthZ server. I will also > provide > > the "overlay" stuff for those with an existing WFLY installation. > > > > This is related with [1]. The server is pretty much done, I've created a > > feature pack on top of KC server-feature-pack. Now I would like to > > distribute something for client apps only. Any thoughts ? > > > > [1] https://github.com/pedroigor/keycloak-authz/issues/21 > > > > ----- Original Message ----- > > From: "Stian Thorgersen" > > To: "Pedro Igor Silva" > > Cc: "keycloak dev" > > Sent: Friday, November 6, 2015 10:53:45 AM > > Subject: Re: [keycloak-dev] Adapter Feature Pack > > > > Assuming what you want is a ZIP with WildFly and client adapter then yes, > > but not sure it works. > > > > What's this for? > > > > On 6 November 2015 at 13:12, Pedro Igor Silva wrote: > > > > > Hi, > > > > > > Is keycloak-adapter-feature-pack the right pack to use to build > > > distributions to deploy client applications only, which require only > the > > > adapters ? In this case, WFLY ones ... > > > > > > Regards. > > > Pedro Igor > > > _______________________________________________ > > > keycloak-dev mailing list > > > keycloak-dev at lists.jboss.org > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/a29c235c/attachment.html From sthorger at redhat.com Fri Nov 6 08:46:27 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 14:46:27 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes In-Reply-To: <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> Message-ID: On 6 November 2015 at 14:29, Erik Mulder wrote: > Thanks for pointing me explicitly to the SPI documentation. Of course that > is exactly what I was looking for in my original question. I don't know how > I overlooked this earlier! Probably I was not picking it up, because of > almost a decade of developing on Spring projects, where this type of thing > works differently. :-) > > I tried a quick test with a jar with one extra ProtocolMapper configured, > put it in the providers folder and it worked like a charm! > > As for the JPA: We'll probably go with your suggestion of the separate > EntityManagerFactory. Indeed there seems to be no way to 'programmatically > extend' the list of entity classes in JPA besides editing or overwriting > the persistence.xml. As you probably know it can be done in Hibernate, but > I guess KeyCloak wants to stick to a generic JPA solution. That said, we > might consider the Hibernate specific solution for our case, since being > able to switch the JPA provider is not a requirement for us. And keeping > the same connection/transaction is a lot easier in reasoning and debugging. > We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? > > > > On 05/11/15 10:52, Stian Thorgersen wrote: > > The way to extend Keycloak is by implementing your own custom providers of > the many SPIs we provide. Some SPIs are more stable (so marked as public) > and others are not (so marked as private). If there are things that you > want to customize that can't be done with an existing SPI then let us know > and we may consider adding additional SPIs. > > On 4 November 2015 at 17:16, Erik Mulder > wrote: > >> Thanks for your response! >> >> Indeed we already did a proof of concept where we added a custom mapper >> the way you described (didn't know it was 'protected' territory :). The >> question is: do we have to override the file >> 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper >> in the original project or is there another way where we don't need to >> touch the original sources and keep all our changes in a separate >> project? And how can we do it such that it stays easy to upgrade to >> newer KeyCloak releases? >> > > Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at > the docs ( > http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) > and examples for other provider ( > https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory > ) > > >> >> As for JPA: it would be easier to integrate with the existing JPA >> project. Again we are wondering whether to start modifying original >> sources (like persistence.xml) or try to 'externalize' our changes >> somehow and integrate them using existing 'hooks' in the system or maybe >> merge projects during build. >> >> Maybe there is no good answer to this and we'll always be having some >> manual merge pains when upgrading to new KeyCloak versions. We just >> wanted to check if there are preferred ways to add functionality with >> the least amount of impact on the original sources. >> > > I initially wanted the ability to add custom entities to the > JpaConnectProvider, but couldn't find a way to define entities > programatically with JPA. To add your own persistence.xml you would have to > define your own implementation of JpaConnectionProvider and change what is > loaded by default (connectionsJpa/provider attribute in > keycloak-server.json). > > Alternative, which is cleaner, but you end up with separate > connection/transaction, is to create your own EntityManagerFactory. If it's > only used by one provider (for example a custom UserFederationProvider) > there's no need to add a connect provider (that's just a way to share one > EntityManagerFactory between multiple providers) and you can just create it > in the MyUserFederationProviderFactory. > > >> >> >> On 04/11/15 15:30, Bill Burke wrote: >> > Custom mappers should be possible. I didn't document it as I wasn't >> > sure if we wanted to make the SPI public. Custom mappers should just >> > follow the Provider SPI and they will be picked up. If you see the >> > META-INF/services/... file in the resources directory of the "services" >> > or "broker" modules you'll see how to set this up. >> > >> > As for extending the JPA datamodel, what you could do is write a new JPA >> > Connections Provider and plug that in. See connections/jpa. I'm not >> > sure how you would handle the liquibase db migration. >> > >> > On 11/4/2015 6:03 AM, Erik Mulder wrote: >> >> Hi everybody, >> >> >> >> Quick intro: I?m part of a development team in The Netherlands that is >> >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >> >> this and will use OpenID Connect to secure our REST services. It?s a >> >> great product and seems to be the only one having both support for all >> >> kinds of security standards and a model and GUI for users and roles. >> >> Thanks for creating it! J >> >> >> >> (if this should be asked instead on the users mailing list, please >> >> correct me and I?ll post it there) >> >> >> >> So far, so good, but we have some extra requirements that do not fit >> >> into the base KeyCloak data model. See below for details if you?re >> >> interested. My question is: what is the preferred way / best practice >> to >> >> extend the functionality of KeyCloak while keeping the impact on the >> >> original sources to a minimum? Of course we could just fork the most >> >> recent version and start hacking away, but we?d like to be able to >> >> upgrade to newer versions of KeyCloak without too much hassle. >> >> Possibilities that we?ve come up with so far: >> >> >> >> 1.Create completely separate modules that will extend the functionality >> >> the way we need. >> >> >> >> 2.Fork on Github, apply custom changes, and try to merge in updates >> from >> >> the master / release branches / tags >> >> >> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >> >> as Truezip >> >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >> >> manipulate zip files by adding/removing/replacing or Shade >> >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine >> multiple >> >> jars to 1 'uber-jar' containing the contents of both and when >> >> overlapping decide on conflicts through configuration. >> >> >> >> Of course number 1 is preferred, but I do not see how to add custom >> >> mappers or JPA entities without making changes in the original module >> >> files. The other options seem like valid alternatives, but maybe there >> >> is better / standard way to do this. So any help / insight / shared >> >> experience on this is much appreciated, thanks! >> >> >> >> Kind regards, >> >> >> >> Erik Mulder >> >> >> >> Senior Software Engineer >> >> >> >> Docdata Payments ? NL >> >> >> >> P.S. Details on why we want to extend the KeyCloak data model: (any >> >> feedback on the contents of this P.S. is also welcome!) >> >> >> >> Our clients are merchants that have several webshops. We manage their >> >> online payments (shopping cart checkout). We want to be able to let a >> >> merchant manage their own users and let a user have different roles for >> >> different webshops within the same merchant. The overall possible roles >> >> are fixed though, no specific roles per merchant. We could create a >> >> separate realm for every merchant, but then we need to duplicate all >> >> roles every time. Furthermore, in KeyCloak there is no concept of a >> role >> >> within a certain context. This is very understandable, since every >> >> situation has it?s own requirements. We did a proof of concept by >> adding >> >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >> >> and adding a custom mapper that can put this information on the Access >> >> token. Worked like a charm! But it does need some changes in the >> >> KeyCloak modules and sources to work, hence the question above. >> >> >> >> >> >> >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/07bd8dad/attachment-0001.html From psilva at redhat.com Fri Nov 6 08:52:29 2015 From: psilva at redhat.com (Pedro Igor Silva) Date: Fri, 6 Nov 2015 08:52:29 -0500 (EST) Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: References: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> <1605280911.5773222.1446815170611.JavaMail.zimbra@redhat.com> <2028970915.5792920.1446816682776.JavaMail.zimbra@redhat.com> Message-ID: <2085282002.5822581.1446817949424.JavaMail.zimbra@redhat.com> ----- Original Message ----- > From: "Stian Thorgersen" > To: "Pedro Igor Silva" > Cc: "keycloak dev" > Sent: Friday, November 6, 2015 11:41:45 AM > Subject: Re: [keycloak-dev] Adapter Feature Pack > > On 6 November 2015 at 14:31, Pedro Igor Silva wrote: > > > Yeah, I saw that. But I was thinking that provide a ready-to-use > > distribution would be useful as well, just for evaluation and testing > > purposes. So you just unzip, start the server and deploy apps. That would > > make things more easier for people evaluating the AuthZ Server. > > > > I'm also considering overlay. I've just tested the adapter-feature-pack > > and yes, it seems it is not working properly. Think I'm going to forget > > about it and just do overlay. > > > > If you want a nice simple way for folks to try it out create a demo dist - > we have our demo dist which includes wildfly + server + adapter + examples. > Single download for everything. > > We just use the overlay to create it. > > > > > > > Btw, aren't Keycloak WFLY adapters based on WFLY 8/9 only ? > > > > WFL9 works on WFL10 AFAIK works != supported :) Will finish the overlay stuff now .... Thanks. > > > > > > ----- Original Message ----- > > From: "Stian Thorgersen" > > To: "Pedro Igor Silva" > > Cc: "keycloak dev" > > Sent: Friday, November 6, 2015 11:14:46 AM > > Subject: Re: [keycloak-dev] Adapter Feature Pack > > > > For clients you should provide an overlay zip that can be installed into > > existing WildFly. That's what we do for Keycloak. Reasoning behind this is > > simple, users want to add adapter to the version of WildFly they are using. > > They may have configuration changes or deployments added. They may also > > have modules or even other extensions loaded. > > > > Ideally there should be a mechanism provided by WildFly that enables > > installing a feature pack into an existing WildFly, but there isn't. That's > > why we have a ZIP + a installation CLI script instead. Take a look at > > > > http://keycloak.github.io/docs/userguide/keycloak-server/html/ch08.html#jboss-adapter > > . > > > > On 6 November 2015 at 14:06, Pedro Igor Silva wrote: > > > > > Would like to provide a ready-to-use WFLY distribution for clients with > > > the necessary stuff to integrate with the AuthZ server. I will also > > provide > > > the "overlay" stuff for those with an existing WFLY installation. > > > > > > This is related with [1]. The server is pretty much done, I've created a > > > feature pack on top of KC server-feature-pack. Now I would like to > > > distribute something for client apps only. Any thoughts ? > > > > > > [1] https://github.com/pedroigor/keycloak-authz/issues/21 > > > > > > ----- Original Message ----- > > > From: "Stian Thorgersen" > > > To: "Pedro Igor Silva" > > > Cc: "keycloak dev" > > > Sent: Friday, November 6, 2015 10:53:45 AM > > > Subject: Re: [keycloak-dev] Adapter Feature Pack > > > > > > Assuming what you want is a ZIP with WildFly and client adapter then yes, > > > but not sure it works. > > > > > > What's this for? > > > > > > On 6 November 2015 at 13:12, Pedro Igor Silva wrote: > > > > > > > Hi, > > > > > > > > Is keycloak-adapter-feature-pack the right pack to use to build > > > > distributions to deploy client applications only, which require only > > the > > > > adapters ? In this case, WFLY ones ... > > > > > > > > Regards. > > > > Pedro Igor > > > > _______________________________________________ > > > > keycloak-dev mailing list > > > > keycloak-dev at lists.jboss.org > > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > > > > > > From sthorger at redhat.com Fri Nov 6 09:04:01 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 15:04:01 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: <563CACE4.7050909@redhat.com> References: <563B3371.8040402@redhat.com> <563B7FBF.8060701@redhat.com> <563CACE4.7050909@redhat.com> Message-ID: I think this is something to look at next year. We don't have the resources to do it properly now. My vote is to add it to the agenda for F2F. On 6 November 2015 at 14:36, Marek Posolda wrote: > On 05/11/15 17:11, Bill Burke wrote: > > > > On 11/5/2015 5:46 AM, Marek Posolda wrote: > >> I wonder if we should improve handling of custom attributes in > >> freemarker templates and their validation. Let's assume that I want to > >> add custom attribute "birthday" and I want to add it to all screens > >> where user can create/edit account. I can see those issues: > >> > >> - Admin will need to edit 3 separate freemarker templates (registration, > >> account management, update profile page) to have this attribute > >> displayed on all those places. > >> > > we've discussed this before. The problem is formatting in each of the > > UIs. Often ordering of attributes is important. Grouping of attributes > > is a must too. > > > > i.e. Address, State, Country, Postal Code > > > > These 4 attribute must be grouped together, and address must come before > > state, and so on. > > > > You might also have a "Billing Address" group that needs to come after > > Home Address group. > I am not seeing an issue with ordering? The same order you configure in > admin console, the same is used in template. It's classic sorted list, > not special tricks needed IMO. The grouping some attributes can be > easily addressed too IMO. > > > > So, we'd have an automatic way for displaying attributes, then the > > developer would think "that looks like shit" and want to format things > > himself. Stian seems to think that CSS will solve this problem, but I'm > > not convinced. > CSS can do a hell of a lot more than you think it can ;) But, I agree it can't always cover everything - that's why I've been saying we need to be able to define templates, but at a smaller granularity. Having to define the whole update-profile page just to add an attribute is not very elegant. > I agree we can't address all possible sort of issues, but IMO it's good > to have something, which will address 90% of cases? We can provide types > (like combobox "Gender" with values "male" / "female" ). We can also > handle multivalued attributes. And most importantly we can provide > validations for all 3 screens. People won't need to write their own Java > validators when they want something simple like ensure "mandatory" field > present or validate with regex. They will still have possibility to > inject the custom validator if they want it. > > At least validation is must IMO as there is no way to add custom > validation for account mgmt or update profile right now. > > What we can't easily do is client-side validation and stuff for > display/hide something based on value of other attributes (For example > display attribute "Favourite car sign" just if selected gender is "male" > ). But not sure if this is often requirement. > > I am seeing lot of similarity with your kc-provider-config directive in > admin console. It can't address all "generic" sort of things and > client-side stuff etc. But for most cases, it's sufficient. > > > > Also the look and feel could be quite different between registration, > > update profile, account management, and the admin profile. IMO, it > > would end up being easier to just edit the freemarker pages directly > > than to have to define each attribute and how it is grouped, ordered, > > and displayed on each of the pages within an admin console UI. > > > > > There could be 3 checkboxes where can admin select if he wants to add > field on registration, account mgmt and update profile. I am not sure if > we need anything for admin console as we already have "Attributes" tab > for users, which is almost ok IMO (we don't have nice solution for > multivalued attributes). > > Marek > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/dc54a030/attachment.html From mstrukel at redhat.com Fri Nov 6 09:18:07 2015 From: mstrukel at redhat.com (Marko Strukelj) Date: Fri, 6 Nov 2015 15:18:07 +0100 Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> References: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> Message-ID: I haven't been looking at this part for a while but this reminds me ... I think keycloak-adapter-feature-pack is not used for any distribution at all. server-dist uses keycloak-server-feature-pack, and server overlay for WF9 extracts modules from there. These two are the only ones using a feature pack approach IIRC. server-demo and adapter distros use maven + ant approach to generate modules - don't rely on a any feature pack. On Fri, Nov 6, 2015 at 1:12 PM, Pedro Igor Silva wrote: > Hi, > > Is keycloak-adapter-feature-pack the right pack to use to build > distributions to deploy client applications only, which require only the > adapters ? In this case, WFLY ones ... > > Regards. > Pedro Igor > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/57ae02c3/attachment.html From mstrukel at redhat.com Fri Nov 6 09:19:29 2015 From: mstrukel at redhat.com (Marko Strukelj) Date: Fri, 6 Nov 2015 15:19:29 +0100 Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: References: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> <1605280911.5773222.1446815170611.JavaMail.zimbra@redhat.com> <2028970915.5792920.1446816682776.JavaMail.zimbra@redhat.com> Message-ID: On Fri, Nov 6, 2015 at 2:41 PM, Stian Thorgersen wrote: > > On 6 November 2015 at 14:31, Pedro Igor Silva wrote: > >> Yeah, I saw that. But I was thinking that provide a ready-to-use >> distribution would be useful as well, just for evaluation and testing >> purposes. So you just unzip, start the server and deploy apps. That would >> make things more easier for people evaluating the AuthZ Server. > > >> I'm also considering overlay. I've just tested the adapter-feature-pack >> and yes, it seems it is not working properly. Think I'm going to forget >> about it and just do overlay. >> > > If you want a nice simple way for folks to try it out create a demo dist - > we have our demo dist which includes wildfly + server + adapter + examples. > Single download for everything. > > We just use the overlay to create it. > > > >> >> Btw, aren't Keycloak WFLY adapters based on WFLY 8/9 only ? >> > > WFL9 works on WFL10 AFAIK > Correct. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/06f1b9e1/attachment.html From mposolda at redhat.com Fri Nov 6 09:21:37 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 6 Nov 2015 15:21:37 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: References: <563B3371.8040402@redhat.com> <563B7FBF.8060701@redhat.com> <563CACE4.7050909@redhat.com> Message-ID: <563CB771.4030708@redhat.com> Sure, if it's not a blocker for jboss.org team to not have it in 1.0 ? I guess dynamic templates can be postponed, but at least missing validation for account mgmt and update profile is quite a gap IMO. I've discussed this with Vlasta a week or two ago (and that's one of main reasons to write this email btv :-) ) Marek On 06/11/15 15:04, Stian Thorgersen wrote: > I think this is something to look at next year. We don't have the > resources to do it properly now. My vote is to add it to the agenda > for F2F. > > On 6 November 2015 at 14:36, Marek Posolda > wrote: > > On 05/11/15 17:11, Bill Burke wrote: > > > > On 11/5/2015 5:46 AM, Marek Posolda wrote: > >> I wonder if we should improve handling of custom attributes in > >> freemarker templates and their validation. Let's assume that I > want to > >> add custom attribute "birthday" and I want to add it to all screens > >> where user can create/edit account. I can see those issues: > >> > >> - Admin will need to edit 3 separate freemarker templates > (registration, > >> account management, update profile page) to have this attribute > >> displayed on all those places. > >> > > we've discussed this before. The problem is formatting in each > of the > > UIs. Often ordering of attributes is important. Grouping of > attributes > > is a must too. > > > > i.e. Address, State, Country, Postal Code > > > > These 4 attribute must be grouped together, and address must > come before > > state, and so on. > > > > You might also have a "Billing Address" group that needs to come > after > > Home Address group. > I am not seeing an issue with ordering? The same order you > configure in > admin console, the same is used in template. It's classic sorted list, > not special tricks needed IMO. The grouping some attributes can be > easily addressed too IMO. > > > > So, we'd have an automatic way for displaying attributes, then the > > developer would think "that looks like shit" and want to format > things > > himself. Stian seems to think that CSS will solve this problem, > but I'm > > not convinced. > > > CSS can do a hell of a lot more than you think it can ;) > > But, I agree it can't always cover everything - that's why I've been > saying we need to be able to define templates, but at a smaller > granularity. Having to define the whole update-profile page just to > add an attribute is not very elegant. > > I agree we can't address all possible sort of issues, but IMO it's > good > to have something, which will address 90% of cases? We can provide > types > (like combobox "Gender" with values "male" / "female" ). We can also > handle multivalued attributes. And most importantly we can provide > validations for all 3 screens. People won't need to write their > own Java > validators when they want something simple like ensure "mandatory" > field > present or validate with regex. They will still have possibility to > inject the custom validator if they want it. > > At least validation is must IMO as there is no way to add custom > validation for account mgmt or update profile right now. > > What we can't easily do is client-side validation and stuff for > display/hide something based on value of other attributes (For example > display attribute "Favourite car sign" just if selected gender is > "male" > ). But not sure if this is often requirement. > > I am seeing lot of similarity with your kc-provider-config > directive in > admin console. It can't address all "generic" sort of things and > client-side stuff etc. But for most cases, it's sufficient. > > > > Also the look and feel could be quite different between > registration, > > update profile, account management, and the admin profile. IMO, it > > would end up being easier to just edit the freemarker pages directly > > than to have to define each attribute and how it is grouped, > ordered, > > and displayed on each of the pages within an admin console UI. > > > > > There could be 3 checkboxes where can admin select if he wants to add > field on registration, account mgmt and update profile. I am not > sure if > we need anything for admin console as we already have "Attributes" tab > for users, which is almost ok IMO (we don't have nice solution for > multivalued attributes). > > Marek > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/78596118/attachment-0001.html From sthorger at redhat.com Fri Nov 6 09:22:14 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 15:22:14 +0100 Subject: [keycloak-dev] Adapter Feature Pack In-Reply-To: References: <1624131288.5732864.1446811684065.JavaMail.zimbra@redhat.com> <1432492465.5734040.1446811931935.JavaMail.zimbra@redhat.com> Message-ID: On 6 November 2015 at 15:18, Marko Strukelj wrote: > I haven't been looking at this part for a while but this reminds me ... I > think keycloak-adapter-feature-pack is not used for any distribution at all. > WildFly Swarm uses adapter feature pack AFAIK > > server-dist uses keycloak-server-feature-pack, and server overlay for WF9 > extracts modules from there. These two are the only ones using a feature > pack approach IIRC. > > server-demo and adapter distros use maven + ant approach to generate > modules - don't rely on a any feature pack. > > On Fri, Nov 6, 2015 at 1:12 PM, Pedro Igor Silva > wrote: > >> Hi, >> >> Is keycloak-adapter-feature-pack the right pack to use to build >> distributions to deploy client applications only, which require only the >> adapters ? In this case, WFLY ones ... >> >> Regards. >> Pedro Igor >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/8873cd4b/attachment.html From sthorger at redhat.com Fri Nov 6 09:23:37 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 15:23:37 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: <563CB771.4030708@redhat.com> References: <563B3371.8040402@redhat.com> <563B7FBF.8060701@redhat.com> <563CACE4.7050909@redhat.com> <563CB771.4030708@redhat.com> Message-ID: You're right validation is required now. I feel a new Validation SPI coming up? On 6 November 2015 at 15:21, Marek Posolda wrote: > Sure, if it's not a blocker for jboss.org team to not have it in 1.0 ? I > guess dynamic templates can be postponed, but at least missing validation > for account mgmt and update profile is quite a gap IMO. > > I've discussed this with Vlasta a week or two ago (and that's one of main > reasons to write this email btv :-) ) > > Marek > > > On 06/11/15 15:04, Stian Thorgersen wrote: > > I think this is something to look at next year. We don't have the > resources to do it properly now. My vote is to add it to the agenda for F2F. > > On 6 November 2015 at 14:36, Marek Posolda wrote: > >> On 05/11/15 17:11, Bill Burke wrote: >> > >> > On 11/5/2015 5:46 AM, Marek Posolda wrote: >> >> I wonder if we should improve handling of custom attributes in >> >> freemarker templates and their validation. Let's assume that I want to >> >> add custom attribute "birthday" and I want to add it to all screens >> >> where user can create/edit account. I can see those issues: >> >> >> >> - Admin will need to edit 3 separate freemarker templates >> (registration, >> >> account management, update profile page) to have this attribute >> >> displayed on all those places. >> >> >> > we've discussed this before. The problem is formatting in each of the >> > UIs. Often ordering of attributes is important. Grouping of attributes >> > is a must too. >> > >> > i.e. Address, State, Country, Postal Code >> > >> > These 4 attribute must be grouped together, and address must come before >> > state, and so on. >> > >> > You might also have a "Billing Address" group that needs to come after >> > Home Address group. >> I am not seeing an issue with ordering? The same order you configure in >> admin console, the same is used in template. It's classic sorted list, >> not special tricks needed IMO. The grouping some attributes can be >> easily addressed too IMO. >> > >> > So, we'd have an automatic way for displaying attributes, then the >> > developer would think "that looks like shit" and want to format things >> > himself. Stian seems to think that CSS will solve this problem, but I'm >> > not convinced. >> > > CSS can do a hell of a lot more than you think it can ;) > > But, I agree it can't always cover everything - that's why I've been > saying we need to be able to define templates, but at a smaller > granularity. Having to define the whole update-profile page just to add an > attribute is not very elegant. > > >> I agree we can't address all possible sort of issues, but IMO it's good >> to have something, which will address 90% of cases? We can provide types >> (like combobox "Gender" with values "male" / "female" ). We can also >> handle multivalued attributes. And most importantly we can provide >> validations for all 3 screens. People won't need to write their own Java >> validators when they want something simple like ensure "mandatory" field >> present or validate with regex. They will still have possibility to >> inject the custom validator if they want it. >> >> At least validation is must IMO as there is no way to add custom >> validation for account mgmt or update profile right now. >> >> What we can't easily do is client-side validation and stuff for >> display/hide something based on value of other attributes (For example >> display attribute "Favourite car sign" just if selected gender is "male" >> ). But not sure if this is often requirement. >> >> I am seeing lot of similarity with your kc-provider-config directive in >> admin console. It can't address all "generic" sort of things and >> client-side stuff etc. But for most cases, it's sufficient. >> > >> > Also the look and feel could be quite different between registration, >> > update profile, account management, and the admin profile. IMO, it >> > would end up being easier to just edit the freemarker pages directly >> > than to have to define each attribute and how it is grouped, ordered, >> > and displayed on each of the pages within an admin console UI. >> > >> > >> There could be 3 checkboxes where can admin select if he wants to add >> field on registration, account mgmt and update profile. I am not sure if >> we need anything for admin console as we already have "Attributes" tab >> for users, which is almost ok IMO (we don't have nice solution for >> multivalued attributes). >> >> Marek >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/80345e36/attachment.html From sthorger at redhat.com Fri Nov 6 09:23:58 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 15:23:58 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: References: <563B3371.8040402@redhat.com> <563B7FBF.8060701@redhat.com> <563CACE4.7050909@redhat.com> <563CB771.4030708@redhat.com> Message-ID: Password policies needs to be made into an SPI as well On 6 November 2015 at 15:23, Stian Thorgersen wrote: > You're right validation is required now. I feel a new Validation SPI > coming up? > > On 6 November 2015 at 15:21, Marek Posolda wrote: > >> Sure, if it's not a blocker for jboss.org team to not have it in 1.0 ? I >> guess dynamic templates can be postponed, but at least missing validation >> for account mgmt and update profile is quite a gap IMO. >> >> I've discussed this with Vlasta a week or two ago (and that's one of main >> reasons to write this email btv :-) ) >> >> Marek >> >> >> On 06/11/15 15:04, Stian Thorgersen wrote: >> >> I think this is something to look at next year. We don't have the >> resources to do it properly now. My vote is to add it to the agenda for F2F. >> >> On 6 November 2015 at 14:36, Marek Posolda wrote: >> >>> On 05/11/15 17:11, Bill Burke wrote: >>> > >>> > On 11/5/2015 5:46 AM, Marek Posolda wrote: >>> >> I wonder if we should improve handling of custom attributes in >>> >> freemarker templates and their validation. Let's assume that I want to >>> >> add custom attribute "birthday" and I want to add it to all screens >>> >> where user can create/edit account. I can see those issues: >>> >> >>> >> - Admin will need to edit 3 separate freemarker templates >>> (registration, >>> >> account management, update profile page) to have this attribute >>> >> displayed on all those places. >>> >> >>> > we've discussed this before. The problem is formatting in each of the >>> > UIs. Often ordering of attributes is important. Grouping of attributes >>> > is a must too. >>> > >>> > i.e. Address, State, Country, Postal Code >>> > >>> > These 4 attribute must be grouped together, and address must come >>> before >>> > state, and so on. >>> > >>> > You might also have a "Billing Address" group that needs to come after >>> > Home Address group. >>> I am not seeing an issue with ordering? The same order you configure in >>> admin console, the same is used in template. It's classic sorted list, >>> not special tricks needed IMO. The grouping some attributes can be >>> easily addressed too IMO. >>> > >>> > So, we'd have an automatic way for displaying attributes, then the >>> > developer would think "that looks like shit" and want to format things >>> > himself. Stian seems to think that CSS will solve this problem, but >>> I'm >>> > not convinced. >>> >> >> CSS can do a hell of a lot more than you think it can ;) >> >> But, I agree it can't always cover everything - that's why I've been >> saying we need to be able to define templates, but at a smaller >> granularity. Having to define the whole update-profile page just to add an >> attribute is not very elegant. >> >> >>> I agree we can't address all possible sort of issues, but IMO it's good >>> to have something, which will address 90% of cases? We can provide types >>> (like combobox "Gender" with values "male" / "female" ). We can also >>> handle multivalued attributes. And most importantly we can provide >>> validations for all 3 screens. People won't need to write their own Java >>> validators when they want something simple like ensure "mandatory" field >>> present or validate with regex. They will still have possibility to >>> inject the custom validator if they want it. >>> >>> At least validation is must IMO as there is no way to add custom >>> validation for account mgmt or update profile right now. >>> >>> What we can't easily do is client-side validation and stuff for >>> display/hide something based on value of other attributes (For example >>> display attribute "Favourite car sign" just if selected gender is "male" >>> ). But not sure if this is often requirement. >>> >>> I am seeing lot of similarity with your kc-provider-config directive in >>> admin console. It can't address all "generic" sort of things and >>> client-side stuff etc. But for most cases, it's sufficient. >>> > >>> > Also the look and feel could be quite different between registration, >>> > update profile, account management, and the admin profile. IMO, it >>> > would end up being easier to just edit the freemarker pages directly >>> > than to have to define each attribute and how it is grouped, ordered, >>> > and displayed on each of the pages within an admin console UI. >>> > >>> > >>> There could be 3 checkboxes where can admin select if he wants to add >>> field on registration, account mgmt and update profile. I am not sure if >>> we need anything for admin console as we already have "Attributes" tab >>> for users, which is almost ok IMO (we don't have nice solution for >>> multivalued attributes). >>> >>> Marek >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/5a7ef65f/attachment-0001.html From mposolda at redhat.com Fri Nov 6 09:28:48 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 6 Nov 2015 15:28:48 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: References: <563B3371.8040402@redhat.com> <563B7FBF.8060701@redhat.com> <563CACE4.7050909@redhat.com> <563CB771.4030708@redhat.com> Message-ID: <563CB920.9070208@redhat.com> +1 Bill already has dynamic validations in Authentication SPI, which is used just for registrations now. Wonder if SPI can be reused/enhanced somehow, so you can easily apply validation on all 3 places. And having the default validators based on mandatory attribute + regex and admin UI for define this. Hopefully something like we have for password policy (or OTP policy) can work for validators too. Marek On 06/11/15 15:23, Stian Thorgersen wrote: > You're right validation is required now. I feel a new Validation SPI > coming up? > > On 6 November 2015 at 15:21, Marek Posolda > wrote: > > Sure, if it's not a blocker for jboss.org team > to not have it in 1.0 ? I guess dynamic templates can be > postponed, but at least missing validation for account mgmt and > update profile is quite a gap IMO. > > I've discussed this with Vlasta a week or two ago (and that's one > of main reasons to write this email btv :-) ) > > Marek > > > On 06/11/15 15:04, Stian Thorgersen wrote: >> I think this is something to look at next year. We don't have the >> resources to do it properly now. My vote is to add it to the >> agenda for F2F. >> >> On 6 November 2015 at 14:36, Marek Posolda > > wrote: >> >> On 05/11/15 17:11, Bill Burke wrote: >> > >> > On 11/5/2015 5:46 AM, Marek Posolda wrote: >> >> I wonder if we should improve handling of custom attributes in >> >> freemarker templates and their validation. Let's assume >> that I want to >> >> add custom attribute "birthday" and I want to add it to >> all screens >> >> where user can create/edit account. I can see those issues: >> >> >> >> - Admin will need to edit 3 separate freemarker templates >> (registration, >> >> account management, update profile page) to have this >> attribute >> >> displayed on all those places. >> >> >> > we've discussed this before. The problem is formatting in >> each of the >> > UIs. Often ordering of attributes is important. Grouping of >> attributes >> > is a must too. >> > >> > i.e. Address, State, Country, Postal Code >> > >> > These 4 attribute must be grouped together, and address >> must come before >> > state, and so on. >> > >> > You might also have a "Billing Address" group that needs to >> come after >> > Home Address group. >> I am not seeing an issue with ordering? The same order you >> configure in >> admin console, the same is used in template. It's classic >> sorted list, >> not special tricks needed IMO. The grouping some attributes >> can be >> easily addressed too IMO. >> > >> > So, we'd have an automatic way for displaying attributes, >> then the >> > developer would think "that looks like shit" and want to >> format things >> > himself. Stian seems to think that CSS will solve this >> problem, but I'm >> > not convinced. >> >> >> CSS can do a hell of a lot more than you think it can ;) >> >> But, I agree it can't always cover everything - that's why I've >> been saying we need to be able to define templates, but at a >> smaller granularity. Having to define the whole update-profile >> page just to add an attribute is not very elegant. >> >> I agree we can't address all possible sort of issues, but IMO >> it's good >> to have something, which will address 90% of cases? We can >> provide types >> (like combobox "Gender" with values "male" / "female" ). We >> can also >> handle multivalued attributes. And most importantly we can >> provide >> validations for all 3 screens. People won't need to write >> their own Java >> validators when they want something simple like ensure >> "mandatory" field >> present or validate with regex. They will still have >> possibility to >> inject the custom validator if they want it. >> >> At least validation is must IMO as there is no way to add custom >> validation for account mgmt or update profile right now. >> >> What we can't easily do is client-side validation and stuff for >> display/hide something based on value of other attributes >> (For example >> display attribute "Favourite car sign" just if selected >> gender is "male" >> ). But not sure if this is often requirement. >> >> I am seeing lot of similarity with your kc-provider-config >> directive in >> admin console. It can't address all "generic" sort of things and >> client-side stuff etc. But for most cases, it's sufficient. >> > >> > Also the look and feel could be quite different between >> registration, >> > update profile, account management, and the admin profile. >> IMO, it >> > would end up being easier to just edit the freemarker pages >> directly >> > than to have to define each attribute and how it is >> grouped, ordered, >> > and displayed on each of the pages within an admin console UI. >> > >> > >> There could be 3 checkboxes where can admin select if he >> wants to add >> field on registration, account mgmt and update profile. I am >> not sure if >> we need anything for admin console as we already have >> "Attributes" tab >> for users, which is almost ok IMO (we don't have nice >> solution for >> multivalued attributes). >> >> Marek >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/cfec7890/attachment.html From mposolda at redhat.com Fri Nov 6 09:29:12 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 6 Nov 2015 15:29:12 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: References: <563B3371.8040402@redhat.com> <563B7FBF.8060701@redhat.com> <563CACE4.7050909@redhat.com> <563CB771.4030708@redhat.com> Message-ID: <563CB938.3060103@redhat.com> +1 On 06/11/15 15:23, Stian Thorgersen wrote: > Password policies needs to be made into an SPI as well > > On 6 November 2015 at 15:23, Stian Thorgersen > wrote: > > You're right validation is required now. I feel a new Validation > SPI coming up? > > On 6 November 2015 at 15:21, Marek Posolda > wrote: > > Sure, if it's not a blocker for jboss.org > team to not have it in 1.0 ? I guess dynamic templates can be > postponed, but at least missing validation for account mgmt > and update profile is quite a gap IMO. > > I've discussed this with Vlasta a week or two ago (and that's > one of main reasons to write this email btv :-) ) > > Marek > > > On 06/11/15 15:04, Stian Thorgersen wrote: >> I think this is something to look at next year. We don't have >> the resources to do it properly now. My vote is to add it to >> the agenda for F2F. >> >> On 6 November 2015 at 14:36, Marek Posolda >> > wrote: >> >> On 05/11/15 17:11, Bill Burke wrote: >> > >> > On 11/5/2015 5:46 AM, Marek Posolda wrote: >> >> I wonder if we should improve handling of custom >> attributes in >> >> freemarker templates and their validation. Let's >> assume that I want to >> >> add custom attribute "birthday" and I want to add it >> to all screens >> >> where user can create/edit account. I can see those >> issues: >> >> >> >> - Admin will need to edit 3 separate freemarker >> templates (registration, >> >> account management, update profile page) to have this >> attribute >> >> displayed on all those places. >> >> >> > we've discussed this before. The problem is formatting >> in each of the >> > UIs. Often ordering of attributes is important. >> Grouping of attributes >> > is a must too. >> > >> > i.e. Address, State, Country, Postal Code >> > >> > These 4 attribute must be grouped together, and address >> must come before >> > state, and so on. >> > >> > You might also have a "Billing Address" group that >> needs to come after >> > Home Address group. >> I am not seeing an issue with ordering? The same order >> you configure in >> admin console, the same is used in template. It's classic >> sorted list, >> not special tricks needed IMO. The grouping some >> attributes can be >> easily addressed too IMO. >> > >> > So, we'd have an automatic way for displaying >> attributes, then the >> > developer would think "that looks like shit" and want >> to format things >> > himself. Stian seems to think that CSS will solve this >> problem, but I'm >> > not convinced. >> >> >> CSS can do a hell of a lot more than you think it can ;) >> >> But, I agree it can't always cover everything - that's why >> I've been saying we need to be able to define templates, but >> at a smaller granularity. Having to define the whole >> update-profile page just to add an attribute is not very elegant. >> >> I agree we can't address all possible sort of issues, but >> IMO it's good >> to have something, which will address 90% of cases? We >> can provide types >> (like combobox "Gender" with values "male" / "female" ). >> We can also >> handle multivalued attributes. And most importantly we >> can provide >> validations for all 3 screens. People won't need to write >> their own Java >> validators when they want something simple like ensure >> "mandatory" field >> present or validate with regex. They will still have >> possibility to >> inject the custom validator if they want it. >> >> At least validation is must IMO as there is no way to add >> custom >> validation for account mgmt or update profile right now. >> >> What we can't easily do is client-side validation and >> stuff for >> display/hide something based on value of other attributes >> (For example >> display attribute "Favourite car sign" just if selected >> gender is "male" >> ). But not sure if this is often requirement. >> >> I am seeing lot of similarity with your >> kc-provider-config directive in >> admin console. It can't address all "generic" sort of >> things and >> client-side stuff etc. But for most cases, it's sufficient. >> > >> > Also the look and feel could be quite different between >> registration, >> > update profile, account management, and the admin >> profile. IMO, it >> > would end up being easier to just edit the freemarker >> pages directly >> > than to have to define each attribute and how it is >> grouped, ordered, >> > and displayed on each of the pages within an admin >> console UI. >> > >> > >> There could be 3 checkboxes where can admin select if he >> wants to add >> field on registration, account mgmt and update profile. I >> am not sure if >> we need anything for admin console as we already have >> "Attributes" tab >> for users, which is almost ok IMO (we don't have nice >> solution for >> multivalued attributes). >> >> Marek >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/c706ece1/attachment-0001.html From sthorger at redhat.com Fri Nov 6 09:31:18 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 6 Nov 2015 15:31:18 +0100 Subject: [keycloak-dev] Custom attributes on registration and account management In-Reply-To: <563CB920.9070208@redhat.com> References: <563B3371.8040402@redhat.com> <563B7FBF.8060701@redhat.com> <563CACE4.7050909@redhat.com> <563CB771.4030708@redhat.com> <563CB920.9070208@redhat.com> Message-ID: JIRA it + add to 1.8 for now On 6 November 2015 at 15:28, Marek Posolda wrote: > +1 > > Bill already has dynamic validations in Authentication SPI, which is used > just for registrations now. Wonder if SPI can be reused/enhanced somehow, > so you can easily apply validation on all 3 places. And having the default > validators based on mandatory attribute + regex and admin UI for define > this. Hopefully something like we have for password policy (or OTP policy) > can work for validators too. > > Marek > > > On 06/11/15 15:23, Stian Thorgersen wrote: > > You're right validation is required now. I feel a new Validation SPI > coming up? > > On 6 November 2015 at 15:21, Marek Posolda wrote: > >> Sure, if it's not a blocker for jboss.org team to not have it in 1.0 ? I >> guess dynamic templates can be postponed, but at least missing validation >> for account mgmt and update profile is quite a gap IMO. >> >> I've discussed this with Vlasta a week or two ago (and that's one of main >> reasons to write this email btv :-) ) >> >> Marek >> >> >> On 06/11/15 15:04, Stian Thorgersen wrote: >> >> I think this is something to look at next year. We don't have the >> resources to do it properly now. My vote is to add it to the agenda for F2F. >> >> On 6 November 2015 at 14:36, Marek Posolda < >> mposolda at redhat.com> wrote: >> >>> On 05/11/15 17:11, Bill Burke wrote: >>> > >>> > On 11/5/2015 5:46 AM, Marek Posolda wrote: >>> >> I wonder if we should improve handling of custom attributes in >>> >> freemarker templates and their validation. Let's assume that I want to >>> >> add custom attribute "birthday" and I want to add it to all screens >>> >> where user can create/edit account. I can see those issues: >>> >> >>> >> - Admin will need to edit 3 separate freemarker templates >>> (registration, >>> >> account management, update profile page) to have this attribute >>> >> displayed on all those places. >>> >> >>> > we've discussed this before. The problem is formatting in each of the >>> > UIs. Often ordering of attributes is important. Grouping of attributes >>> > is a must too. >>> > >>> > i.e. Address, State, Country, Postal Code >>> > >>> > These 4 attribute must be grouped together, and address must come >>> before >>> > state, and so on. >>> > >>> > You might also have a "Billing Address" group that needs to come after >>> > Home Address group. >>> I am not seeing an issue with ordering? The same order you configure in >>> admin console, the same is used in template. It's classic sorted list, >>> not special tricks needed IMO. The grouping some attributes can be >>> easily addressed too IMO. >>> > >>> > So, we'd have an automatic way for displaying attributes, then the >>> > developer would think "that looks like shit" and want to format things >>> > himself. Stian seems to think that CSS will solve this problem, but >>> I'm >>> > not convinced. >>> >> >> CSS can do a hell of a lot more than you think it can ;) >> >> But, I agree it can't always cover everything - that's why I've been >> saying we need to be able to define templates, but at a smaller >> granularity. Having to define the whole update-profile page just to add an >> attribute is not very elegant. >> >> >>> I agree we can't address all possible sort of issues, but IMO it's good >>> to have something, which will address 90% of cases? We can provide types >>> (like combobox "Gender" with values "male" / "female" ). We can also >>> handle multivalued attributes. And most importantly we can provide >>> validations for all 3 screens. People won't need to write their own Java >>> validators when they want something simple like ensure "mandatory" field >>> present or validate with regex. They will still have possibility to >>> inject the custom validator if they want it. >>> >>> At least validation is must IMO as there is no way to add custom >>> validation for account mgmt or update profile right now. >>> >>> What we can't easily do is client-side validation and stuff for >>> display/hide something based on value of other attributes (For example >>> display attribute "Favourite car sign" just if selected gender is "male" >>> ). But not sure if this is often requirement. >>> >>> I am seeing lot of similarity with your kc-provider-config directive in >>> admin console. It can't address all "generic" sort of things and >>> client-side stuff etc. But for most cases, it's sufficient. >>> > >>> > Also the look and feel could be quite different between registration, >>> > update profile, account management, and the admin profile. IMO, it >>> > would end up being easier to just edit the freemarker pages directly >>> > than to have to define each attribute and how it is grouped, ordered, >>> > and displayed on each of the pages within an admin console UI. >>> > >>> > >>> There could be 3 checkboxes where can admin select if he wants to add >>> field on registration, account mgmt and update profile. I am not sure if >>> we need anything for admin console as we already have "Attributes" tab >>> for users, which is almost ok IMO (we don't have nice solution for >>> multivalued attributes). >>> >>> Marek >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151106/c2302175/attachment.html From erik.mulder at docdatapayments.com Sat Nov 7 17:39:08 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Sat, 7 Nov 2015 23:39:08 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> On 06/11/15 14:46, Stian Thorgersen wrote: > We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? I was a little quick to state that with Hibernate you can add extra entity class names besides the one in persistence.xml, since I spotted a few answers on StackOverflow that said it could be done. But they resolve around classpath scanning or using a Spring managed Hibernate. Then I thought: 'if Spring can do it, I can do it too' so I investigated the Hibernate source code 'behind' Persistence.createEntityManagerFactory(unitName, properties). After some digging it turns out it's pretty simple to get extra class names in the configuration. See code sample below. The only problem is that Hibernate will only find classes that are part of the 'main' KeyCloak application, because of the way the Wildfly module system and ClassLoader strategy work. The debugger showed me Hibernate has these 3 class loaders available to look for classes: 1. ModuleClassLoader for Module "deployment.keycloak-server.war:main" from Service Module Loader 2. ModuleClassLoader for Module "org.hibernate:main" from local module loader 3. sun.misc.Launcher$AppClassLoader Number 1 has all other KeyCloak modules in it, so the entity classes from model-jpa will be found, but the wildfly-extensions module is missing, so entities in classes in a jar in the providers folder cannot be found. Now you guys obviously know a lot more about these internals, but as currently configured, it seems to me there is no way to let Hibernate 'see' these extra classes, since only the KeyCloak services module has a dependency on wildfly-extensions. So I think it boils down to these decisions: A. Do you accept a non-pure-JPA way of building the EntityManagerFactory that has some ties to the Hibernate internals? B. If A is no, than we're done. If yes, then you must find some way to get the extra configured classes 'into' Hibernate. You could get the wildfly-extensions module into scope of the Hibernate classloading. There are serveral ways to configure Hibernate classloading or you could flip some switches / dependencies in the module configuration. Another alternative is to create a separate 'dropfolder' besides themes and providers for JPA extensions, like 'models' or so and have that one be on the Hibernate classpath. But I don't know the exact design principles behind KeyCloak or the Wildfly module system. So maybe you have a better solution or maybe you conclude that this is 'not done' in terms of the architecture. Either way, I'd really appreciate some feedback on this and some thoughts on whether this could be a possible addition to KeyCloak in your eyes. Thanks, Erik Current JPA way. No way to 'interfere': emf = Persistence.createEntityManagerFactory(unitName, properties); Alternative Hibernate only way with adding extra entity class names: // Let Hibernate find and parse all 'persistence.xml' files found on the classpath. List persistenceUnits = PersistenceXmlParser.locatePersistenceUnits(properties); // Assume there is only one persistence unit found and that is the one we need. This can be made more robust by checking on the persistence unit name. ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = persistenceUnits.get(0); // Add extra class names. These could come from a 'JPA class name provider' SPI or something alike. persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity", "org.keycloak.models.jpa.entities.MerchantEntity"); // Let Hibernate create an EntityManagerFactory out of the (enriched) persistence unit configuration. emf = Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, properties).build(); On 6 November 2015 at 14:29, Erik Mulder > wrote: Thanks for pointing me explicitly to the SPI documentation. Of course that is exactly what I was looking for in my original question. I don't know how I overlooked this earlier! Probably I was not picking it up, because of almost a decade of developing on Spring projects, where this type of thing works differently. :-) I tried a quick test with a jar with one extra ProtocolMapper configured, put it in the providers folder and it worked like a charm! As for the JPA: We'll probably go with your suggestion of the separate EntityManagerFactory. Indeed there seems to be no way to 'programmatically extend' the list of entity classes in JPA besides editing or overwriting the persistence.xml. As you probably know it can be done in Hibernate, but I guess KeyCloak wants to stick to a generic JPA solution. That said, we might consider the Hibernate specific solution for our case, since being able to switch the JPA provider is not a requirement for us. And keeping the same connection/transaction is a lot easier in reasoning and debugging. We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? On 05/11/15 10:52, Stian Thorgersen wrote: The way to extend Keycloak is by implementing your own custom providers of the many SPIs we provide. Some SPIs are more stable (so marked as public) and others are not (so marked as private). If there are things that you want to customize that can't be done with an existing SPI then let us know and we may consider adding additional SPIs. On 4 November 2015 at 17:16, Erik Mulder <erik.mulder at docdatapayments.com> wrote: Thanks for your response! Indeed we already did a proof of concept where we added a custom mapper the way you described (didn't know it was 'protected' territory :). The question is: do we have to override the file 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper in the original project or is there another way where we don't need to touch the original sources and keep all our changes in a separate project? And how can we do it such that it stays easy to upgrade to newer KeyCloak releases? Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at the docs (http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) and examples for other provider (https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory) As for JPA: it would be easier to integrate with the existing JPA project. Again we are wondering whether to start modifying original sources (like persistence.xml) or try to 'externalize' our changes somehow and integrate them using existing 'hooks' in the system or maybe merge projects during build. Maybe there is no good answer to this and we'll always be having some manual merge pains when upgrading to new KeyCloak versions. We just wanted to check if there are preferred ways to add functionality with the least amount of impact on the original sources. I initially wanted the ability to add custom entities to the JpaConnectProvider, but couldn't find a way to define entities programatically with JPA. To add your own persistence.xml you would have to define your own implementation of JpaConnectionProvider and change what is loaded by default (connectionsJpa/provider attribute in keycloak-server.json). Alternative, which is cleaner, but you end up with separate connection/transaction, is to create your own EntityManagerFactory. If it's only used by one provider (for example a custom UserFederationProvider) there's no need to add a connect provider (that's just a way to share one EntityManagerFactory between multiple providers) and you can just create it in the MyUserFederationProviderFactory. On 04/11/15 15:30, Bill Burke wrote: > Custom mappers should be possible. I didn't document it as I wasn't > sure if we wanted to make the SPI public. Custom mappers should just > follow the Provider SPI and they will be picked up. If you see the > META-INF/services/... file in the resources directory of the "services" > or "broker" modules you'll see how to set this up. > > As for extending the JPA datamodel, what you could do is write a new JPA > Connections Provider and plug that in. See connections/jpa. I'm not > sure how you would handle the liquibase db migration. > > On 11/4/2015 6:03 AM, Erik Mulder wrote: >> Hi everybody, >> >> Quick intro: I?m part of a development team in The Netherlands that is >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >> this and will use OpenID Connect to secure our REST services. It?s a >> great product and seems to be the only one having both support for all >> kinds of security standards and a model and GUI for users and roles. >> Thanks for creating it! J >> >> (if this should be asked instead on the users mailing list, please >> correct me and I?ll post it there) >> >> So far, so good, but we have some extra requirements that do not fit >> into the base KeyCloak data model. See below for details if you?re >> interested. My question is: what is the preferred way / best practice to >> extend the functionality of KeyCloak while keeping the impact on the >> original sources to a minimum? Of course we could just fork the most >> recent version and start hacking away, but we?d like to be able to >> upgrade to newer versions of KeyCloak without too much hassle. >> Possibilities that we?ve come up with so far: >> >> 1.Create completely separate modules that will extend the functionality >> the way we need. >> >> 2.Fork on Github, apply custom changes, and try to merge in updates from >> the master / release branches / tags >> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >> as Truezip >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >> manipulate zip files by adding/removing/replacing or Shade >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple >> jars to 1 'uber-jar' containing the contents of both and when >> overlapping decide on conflicts through configuration. >> >> Of course number 1 is preferred, but I do not see how to add custom >> mappers or JPA entities without making changes in the original module >> files. The other options seem like valid alternatives, but maybe there >> is better / standard way to do this. So any help / insight / shared >> experience on this is much appreciated, thanks! >> >> Kind regards, >> >> Erik Mulder >> >> Senior Software Engineer >> >> Docdata Payments ? NL >> >> P.S. Details on why we want to extend the KeyCloak data model: (any >> feedback on the contents of this P.S. is also welcome!) >> >> Our clients are merchants that have several webshops. We manage their >> online payments (shopping cart checkout). We want to be able to let a >> merchant manage their own users and let a user have different roles for >> different webshops within the same merchant. The overall possible roles >> are fixed though, no specific roles per merchant. We could create a >> separate realm for every merchant, but then we need to duplicate all >> roles every time. Furthermore, in KeyCloak there is no concept of a role >> within a certain context. This is very understandable, since every >> situation has it?s own requirements. We did a proof of concept by adding >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >> and adding a custom mapper that can put this information on the Access >> token. Worked like a charm! But it does need some changes in the >> KeyCloak modules and sources to work, hence the question above. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151107/33d25e05/attachment-0001.html From sthorger at redhat.com Mon Nov 9 07:20:18 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 9 Nov 2015 13:20:18 +0100 Subject: [keycloak-dev] Import proposal Message-ID: Currently we support importing a complete realm definition using the import/export feature. Issues with the current approach is: * Only complete realm - not possible to add to an existing realm * No good feedback if import was successful or not * Use of system properties to initiate the import is not very user friendly * Not very elegant for provisioning. For example a Docker image that want's to bundle some initial setup ends up always running the import of a realm, which is skipped if realm exists To solve this I've come up with the following proposal: Allow dropping representations to be imported into 'standalone/import'. This should support creating a new realm as well as importing into an existing realm. When importing into an existing realm we will have an import strategy that is used to configure what happens if a resource exists (user, role, identity provider, user federtation provider). The import strategies are: * Skip - existing resources are skipped, * Fail - if any resource exists nothing is imported * Overwrite - any existing resources are deleted. The directory will be scanned at startup, but there will also be an option to monitor this directory at runtime. To prevent a file being imported multiple times (also to make sure only one node in a cluster imports) we will have a table in the database that contains what files was imported, from what node, date and result (including a list of what resources where imported, which was not, and stack trace if applicable). The primary key will be the checksum of the file. We will also add marker files (.imported or .failed). The contents of the marker files will be a json object with date imported, outcome (including stack trace if applicable) as well as a complete list of what resources was successfully imported, what where not. The files will also allow resolving system properties and environment variables. For example: { "secret": "${env.MYCLIENT_SECRET}" } This will be very convenient for example with Docker as it would be very easy to create a Docker image that extends ours to add a few clients and users. It will also be convenient for examples as it will make it possible to add the required clients and users to an existing realm. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151109/2153ff2d/attachment.html From sblanc at redhat.com Mon Nov 9 07:35:33 2015 From: sblanc at redhat.com (Sebastien Blanc) Date: Mon, 9 Nov 2015 13:35:33 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: References: Message-ID: That would be really nice indeed ! But are the markers files not enough, instead of also having a table in the DB ? On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen wrote: > Currently we support importing a complete realm definition using the > import/export feature. Issues with the current approach is: > > * Only complete realm - not possible to add to an existing realm > * No good feedback if import was successful or not > * Use of system properties to initiate the import is not very user friendly > * Not very elegant for provisioning. For example a Docker image that > want's to bundle some initial setup ends up always running the import of a > realm, which is skipped if realm exists > > To solve this I've come up with the following proposal: > > Allow dropping representations to be imported into 'standalone/import'. > This should support creating a new realm as well as importing into an > existing realm. When importing into an existing realm we will have an > import strategy that is used to configure what happens if a resource exists > (user, role, identity provider, user federtation provider). The import > strategies are: > > * Skip - existing resources are skipped, > * Fail - if any resource exists nothing is imported > * Overwrite - any existing resources are deleted. > > The directory will be scanned at startup, but there will also be an option > to monitor this directory at runtime. > > To prevent a file being imported multiple times (also to make sure only > one node in a cluster imports) we will have a table in the database that > contains what files was imported, from what node, date and result > (including a list of what resources where imported, which was not, and > stack trace if applicable). The primary key will be the checksum of the > file. We will also add marker files (.imported or file>.failed). The contents of the marker files will be a json object with > date imported, outcome (including stack trace if applicable) as well as a > complete list of what resources was successfully imported, what where not. > > The files will also allow resolving system properties and environment > variables. For example: > > { > "secret": "${env.MYCLIENT_SECRET}" > } > > This will be very convenient for example with Docker as it would be very > easy to create a Docker image that extends ours to add a few clients and > users. > > It will also be convenient for examples as it will make it possible to add > the required clients and users to an existing realm. > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151109/b79701a3/attachment.html From sthorger at redhat.com Mon Nov 9 08:09:45 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 9 Nov 2015 14:09:45 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: References: Message-ID: On 9 November 2015 at 13:35, Sebastien Blanc wrote: > That would be really nice indeed ! > But are the markers files not enough, instead of also having a table in > the DB ? > We need a way to prevent multiple nodes in a cluster to import the same file. For example on Kerberos you end up spinning up multiple instances of the same Docker image. > > > On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen > wrote: > >> Currently we support importing a complete realm definition using the >> import/export feature. Issues with the current approach is: >> >> * Only complete realm - not possible to add to an existing realm >> * No good feedback if import was successful or not >> * Use of system properties to initiate the import is not very user >> friendly >> * Not very elegant for provisioning. For example a Docker image that >> want's to bundle some initial setup ends up always running the import of a >> realm, which is skipped if realm exists >> >> To solve this I've come up with the following proposal: >> >> Allow dropping representations to be imported into 'standalone/import'. >> This should support creating a new realm as well as importing into an >> existing realm. When importing into an existing realm we will have an >> import strategy that is used to configure what happens if a resource exists >> (user, role, identity provider, user federtation provider). The import >> strategies are: >> >> * Skip - existing resources are skipped, >> * Fail - if any resource exists nothing is imported >> * Overwrite - any existing resources are deleted. >> >> The directory will be scanned at startup, but there will also be an >> option to monitor this directory at runtime. >> >> To prevent a file being imported multiple times (also to make sure only >> one node in a cluster imports) we will have a table in the database that >> contains what files was imported, from what node, date and result >> (including a list of what resources where imported, which was not, and >> stack trace if applicable). The primary key will be the checksum of the >> file. We will also add marker files (.imported or > file>.failed). The contents of the marker files will be a json object with >> date imported, outcome (including stack trace if applicable) as well as a >> complete list of what resources was successfully imported, what where not. >> >> The files will also allow resolving system properties and environment >> variables. For example: >> >> { >> "secret": "${env.MYCLIENT_SECRET}" >> } >> >> This will be very convenient for example with Docker as it would be very >> easy to create a Docker image that extends ours to add a few clients and >> users. >> >> It will also be convenient for examples as it will make it possible to >> add the required clients and users to an existing realm. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151109/e73b6657/attachment.html From csnyder at iland.com Mon Nov 9 14:32:58 2015 From: csnyder at iland.com (Cory Snyder) Date: Mon, 9 Nov 2015 19:32:58 +0000 Subject: [keycloak-dev] Require password change on login when AD is the federation provider and pwdLastSet equals 0 Message-ID: <76A3D583-AF5C-4616-B3B8-12B9C33C9CAE@iland.com> Hey guys, Following up on this conversation that took place a couple of months back: http://lists.jboss.org/pipermail/keycloak-dev/2015-September/005286.html. I just had a chance to try the proposed approach of implementing a custom authentication provider that checks the pwdLastSet attribute and sets the update password required action. I believe that this may not be quite as easy as was suggested due to the fact that authentication fails with the default LDAP Federation Provider before a custom execution in the login flow has a chance to check the attribute and set the required action. It seems I would need to implement a custom LDAP Federation Provider that considers authentication successful when the exception referenced in https://issues.jboss.org/browse/KEYCLOAK-1744 is thrown, but also add the required action for updating the password. Is there an easy way to do that or something that I?m missing? Otherwise, I?d be willing to work on a contribution for this issue if you?re willing to have logic that is specific to AD? Thanks, Cory Snyder software engineer USA +1.419.731.3479 UK +44.20.7096.0149 iland.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151109/a5a759a3/attachment-0001.html From mposolda at redhat.com Tue Nov 10 08:27:27 2015 From: mposolda at redhat.com (Marek Posolda) Date: Tue, 10 Nov 2015 14:27:27 +0100 Subject: [keycloak-dev] Require password change on login when AD is the federation provider and pwdLastSet equals 0 In-Reply-To: <76A3D583-AF5C-4616-B3B8-12B9C33C9CAE@iland.com> References: <76A3D583-AF5C-4616-B3B8-12B9C33C9CAE@iland.com> Message-ID: <5641F0BF.6000208@redhat.com> I agree we need some way to address this. Active Directory is widely used and more people asked for that . I've put the comment to https://issues.jboss.org/browse/KEYCLOAK-1744 with possible solution, but it may need changes in UserFederationProvider interface, so the federationProvider is able to propagate the cause why password validation failed (password is expired, user is disabled, or just invalid password was used etc...). As a temporary workaround, you can subclass LDAPFederationProvider and do something on your own. You can override LDAPFederationProvider.validPassword and add updatePassword on User when you detect that reason of password validation failure is expired password. Marek On 09/11/15 20:32, Cory Snyder wrote: > Hey guys, > > Following up on this conversation that took place a couple of months > back: > http://lists.jboss.org/pipermail/keycloak-dev/2015-September/005286.html. > I just had a chance to try the proposed approach of implementing a > custom authentication provider that checks the pwdLastSet attribute > and sets the update password required action. I believe that this may > not be quite as easy as was suggested due to the fact that > authentication fails with the default LDAP Federation Provider before > a custom execution in the login flow has a chance to check the > attribute and set the required action. It seems I would need to > implement a custom LDAP Federation Provider that considers > authentication successful when the exception referenced in > https://issues.jboss.org/browse/KEYCLOAK-1744 is thrown, but also add > the required action for updating the password. Is there an easy way to > do that or something that I?m missing? Otherwise, I?d be willing to > work on a contribution for this issue if you?re willing to have logic > that is specific to AD? > > Thanks, > > Cory Snyder > software engineer > USA +1.419.731.3479 UK +44.20.7096.0149 iland.com > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151110/c466f975/attachment.html From mposolda at redhat.com Tue Nov 10 08:32:31 2015 From: mposolda at redhat.com (Marek Posolda) Date: Tue, 10 Nov 2015 14:32:31 +0100 Subject: [keycloak-dev] Require password change on login when AD is the federation provider and pwdLastSet equals 0 In-Reply-To: <5641F0BF.6000208@redhat.com> References: <76A3D583-AF5C-4616-B3B8-12B9C33C9CAE@iland.com> <5641F0BF.6000208@redhat.com> Message-ID: <5641F1EF.3070402@redhat.com> Btv. I think it should be possible to do it with authenticator as well, but you need to configure authentication flow correctly. You will have your custom authenticator, which will check pwdLastSet and if it's 0, it will put the requiredAction on user and at the same time it will set user as authenticated. It would need to be ALTERNATIVE authenticator used in browser flow before "Forms" authenticator (Forms authenticator is that one which displays login form with username/password and optionally OTP). That way, forms authenticator won't be used and username/password form won't be displayed. It will go directly to requiredActions and user will be asked to update password. Marek 0On 10/11/15 14:27, Marek Posolda wrote: > I agree we need some way to address this. Active Directory is widely > used and more people asked for that . I've put the comment to > https://issues.jboss.org/browse/KEYCLOAK-1744 with possible solution, > but it may need changes in UserFederationProvider interface, so the > federationProvider is able to propagate the cause why password > validation failed (password is expired, user is disabled, or just > invalid password was used etc...). > > As a temporary workaround, you can subclass LDAPFederationProvider and > do something on your own. You can override > LDAPFederationProvider.validPassword and add updatePassword on User > when you detect that reason of password validation failure is expired > password. > > Marek > > On 09/11/15 20:32, Cory Snyder wrote: >> Hey guys, >> >> Following up on this conversation that took place a couple of months >> back: >> http://lists.jboss.org/pipermail/keycloak-dev/2015-September/005286.html. >> I just had a chance to try the proposed approach of implementing a >> custom authentication provider that checks the pwdLastSet attribute >> and sets the update password required action. I believe that this may >> not be quite as easy as was suggested due to the fact that >> authentication fails with the default LDAP Federation Provider before >> a custom execution in the login flow has a chance to check the >> attribute and set the required action. It seems I would need to >> implement a custom LDAP Federation Provider that considers >> authentication successful when the exception referenced in >> https://issues.jboss.org/browse/KEYCLOAK-1744 is thrown, but also add >> the required action for updating the password. Is there an easy way >> to do that or something that I?m missing? Otherwise, I?d be willing >> to work on a contribution for this issue if you?re willing to have >> logic that is specific to AD? >> >> Thanks, >> >> Cory Snyder >> software engineer >> USA +1.419.731.3479 UK +44.20.7096.0149 iland.com >> >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151110/dad030f6/attachment.html From csnyder at iland.com Tue Nov 10 08:40:45 2015 From: csnyder at iland.com (Cory Snyder) Date: Tue, 10 Nov 2015 13:40:45 +0000 Subject: [keycloak-dev] Require password change on login when AD is the federation provider and pwdLastSet equals 0 In-Reply-To: <5641F1EF.3070402@redhat.com> References: <76A3D583-AF5C-4616-B3B8-12B9C33C9CAE@iland.com> <5641F0BF.6000208@redhat.com> <5641F1EF.3070402@redhat.com> Message-ID: Thanks for the suggestions, Marek. I think that overriding the LDAPFederationProvider will be the way to go. The problem that I am seeing which is preventing me from doing this in just the authenticator is that I have no way to authenticate the user in order to allow them to update the password unless I also make changes to the LDAPFederationProvider. As you mentioned, the LDAPFederationProvider simply returns false from the validCredentials method when the pwdLastSet attribute equals 0. If I don?t authenticate the user with their current password beforehand, it would allow anyone with the username to change the user?s password. Or am I overlooking something? Thanks, Cory Snyder software engineer USA +1.419.731.3479 UK +44.20.7096.0149 iland.com On Nov 10, 2015, at 8:32 AM, Marek Posolda > wrote: Btv. I think it should be possible to do it with authenticator as well, but you need to configure authentication flow correctly. You will have your custom authenticator, which will check pwdLastSet and if it's 0, it will put the requiredAction on user and at the same time it will set user as authenticated. It would need to be ALTERNATIVE authenticator used in browser flow before "Forms" authenticator (Forms authenticator is that one which displays login form with username/password and optionally OTP). That way, forms authenticator won't be used and username/password form won't be displayed. It will go directly to requiredActions and user will be asked to update password. Marek 0On 10/11/15 14:27, Marek Posolda wrote: I agree we need some way to address this. Active Directory is widely used and more people asked for that . I've put the comment to https://issues.jboss.org/browse/KEYCLOAK-1744 with possible solution, but it may need changes in UserFederationProvider interface, so the federationProvider is able to propagate the cause why password validation failed (password is expired, user is disabled, or just invalid password was used etc...). As a temporary workaround, you can subclass LDAPFederationProvider and do something on your own. You can override LDAPFederationProvider.validPassword and add updatePassword on User when you detect that reason of password validation failure is expired password. Marek On 09/11/15 20:32, Cory Snyder wrote: Hey guys, Following up on this conversation that took place a couple of months back: http://lists.jboss.org/pipermail/keycloak-dev/2015-September/005286.html. I just had a chance to try the proposed approach of implementing a custom authentication provider that checks the pwdLastSet attribute and sets the update password required action. I believe that this may not be quite as easy as was suggested due to the fact that authentication fails with the default LDAP Federation Provider before a custom execution in the login flow has a chance to check the attribute and set the required action. It seems I would need to implement a custom LDAP Federation Provider that considers authentication successful when the exception referenced in https://issues.jboss.org/browse/KEYCLOAK-1744 is thrown, but also add the required action for updating the password. Is there an easy way to do that or something that I?m missing? Otherwise, I?d be willing to work on a contribution for this issue if you?re willing to have logic that is specific to AD? Thanks, Cory Snyder software engineer USA +1.419.731.3479 UK +44.20.7096.0149 iland.com _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151110/1d14976a/attachment-0001.html From mposolda at redhat.com Tue Nov 10 09:07:54 2015 From: mposolda at redhat.com (Marek Posolda) Date: Tue, 10 Nov 2015 15:07:54 +0100 Subject: [keycloak-dev] Require password change on login when AD is the federation provider and pwdLastSet equals 0 In-Reply-To: References: <76A3D583-AF5C-4616-B3B8-12B9C33C9CAE@iland.com> <5641F0BF.6000208@redhat.com> <5641F1EF.3070402@redhat.com> Message-ID: <5641FA3A.9090506@redhat.com> Yep, agree it's better to handle at LDAPFederationProvider level. Marek On 10/11/15 14:40, Cory Snyder wrote: > Thanks for the suggestions, Marek. I think that overriding the > LDAPFederationProvider will be the way to go. The problem that I am > seeing which is preventing me from doing this in just the > authenticator is that I have no way to authenticate the user in order > to allow them to update the password unless I also make changes to the > LDAPFederationProvider. As you mentioned, the LDAPFederationProvider > simply returns false from the validCredentials method when the > pwdLastSet attribute equals 0. If I don?t authenticate the user with > their current password beforehand, it would allow anyone with the > username to change the user?s password. Or am I overlooking something? > > Thanks, > > Cory Snyder > software engineer > USA +1.419.731.3479 UK +44.20.7096.0149 iland.com > > >> On Nov 10, 2015, at 8:32 AM, Marek Posolda > > wrote: >> >> Btv. I think it should be possible to do it with authenticator as >> well, but you need to configure authentication flow correctly. You >> will have your custom authenticator, which will check pwdLastSet and >> if it's 0, it will put the requiredAction on user and at the same >> time it will set user as authenticated. It would need to be >> ALTERNATIVE authenticator used in browser flow before "Forms" >> authenticator (Forms authenticator is that one which displays login >> form with username/password and optionally OTP). That way, forms >> authenticator won't be used and username/password form won't be >> displayed. It will go directly to requiredActions and user will be >> asked to update password. >> >> Marek >> >> 0On 10/11/15 14:27, Marek Posolda wrote: >>> I agree we need some way to address this. Active Directory is widely >>> used and more people asked for that . I've put the comment to >>> https://issues.jboss.org/browse/KEYCLOAK-1744 with possible >>> solution, but it may need changes in UserFederationProvider >>> interface, so the federationProvider is able to propagate the cause >>> why password validation failed (password is expired, user is >>> disabled, or just invalid password was used etc...). >>> >>> As a temporary workaround, you can subclass LDAPFederationProvider >>> and do something on your own. You can override >>> LDAPFederationProvider.validPassword and add updatePassword on User >>> when you detect that reason of password validation failure is >>> expired password. >>> >>> Marek >>> >>> On 09/11/15 20:32, Cory Snyder wrote: >>>> Hey guys, >>>> >>>> Following up on this conversation that took place a couple of >>>> months back: >>>> http://lists.jboss.org/pipermail/keycloak-dev/2015-September/005286.html. >>>> I just had a chance to try the proposed approach of implementing a >>>> custom authentication provider that checks the pwdLastSet attribute >>>> and sets the update password required action. I believe that this >>>> may not be quite as easy as was suggested due to the fact that >>>> authentication fails with the default LDAP Federation Provider >>>> before a custom execution in the login flow has a chance to check >>>> the attribute and set the required action. It seems I would need to >>>> implement a custom LDAP Federation Provider that considers >>>> authentication successful when the exception referenced in >>>> https://issues.jboss.org/browse/KEYCLOAK-1744 is thrown, but also >>>> add the required action for updating the password. Is there an easy >>>> way to do that or something that I?m missing? Otherwise, I?d be >>>> willing to work on a contribution for this issue if you?re willing >>>> to have logic that is specific to AD? >>>> >>>> Thanks, >>>> >>>> Cory Snyder >>>> software engineer >>>> USA +1.419.731.3479 UK +44.20.7096.0149 iland.com >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151110/bacbe934/attachment.html From mposolda at redhat.com Tue Nov 10 10:11:59 2015 From: mposolda at redhat.com (Marek Posolda) Date: Tue, 10 Nov 2015 16:11:59 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: References: Message-ID: <5642093F.6030308@redhat.com> On 09/11/15 14:09, Stian Thorgersen wrote: > > > On 9 November 2015 at 13:35, Sebastien Blanc > wrote: > > That would be really nice indeed ! > But are the markers files not enough, instead of also having a > table in the DB ? > > > We need a way to prevent multiple nodes in a cluster to import the > same file. For example on Kerberos you end up spinning up multiple > instances of the same Docker image. I bet you meant 'Kubernetes' :-) +1 for the improvements. Besides those I think that earlier or later, we will need to solve long-running export+import where you want to import 100.000 users. As I mentioned in another mail few weeks ago, we can have: 1) Table with the progress (51.000 users already imported, around 49.000 remaining etc.) 2) Concurrency and dividing the work among cluster nodes (Node1 will import 50.000 users and node2 another 50.000 users) 3) Failover (Import won't be completely broken if cluster node crashes after import 90.000, but can continue on other cluster nodes) I think the stuff I did recently for pre-loading offline sessions at startup could be reused for this stuff too and it can handle (2) and (3) . Also it can handle parallel import triggered from more cluster nodes. For example: currently if you trigger kubernetes with 2 cluster nodes, both nodes will start to import same file at the same time because import triggered by node1 is not yet finished before node2 is started, so there is not yet existing DB record that file is already imported. With the stuff I did, just the coordinator (node1) will start the import . Node2 will wait until import triggered by node1 is finished, but at the same time it can "help" to import some users (pages) if coordinator asks him to do so. This impl is based on infinispan distributed executor service http://infinispan.org/docs/5.3.x/user_guide/user_guide.html#_distributed_execution_framework . Marek > > > On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen > > wrote: > > Currently we support importing a complete realm definition > using the import/export feature. Issues with the current > approach is: > > * Only complete realm - not possible to add to an existing realm > * No good feedback if import was successful or not > * Use of system properties to initiate the import is not very > user friendly > * Not very elegant for provisioning. For example a Docker > image that want's to bundle some initial setup ends up always > running the import of a realm, which is skipped if realm exists > > To solve this I've come up with the following proposal: > > Allow dropping representations to be imported into > 'standalone/import'. This should support creating a new realm > as well as importing into an existing realm. When importing > into an existing realm we will have an import strategy that is > used to configure what happens if a resource exists (user, > role, identity provider, user federtation provider). The > import strategies are: > > * Skip - existing resources are skipped, > * Fail - if any resource exists nothing is imported > * Overwrite - any existing resources are deleted. > > The directory will be scanned at startup, but there will also > be an option to monitor this directory at runtime. > > To prevent a file being imported multiple times (also to make > sure only one node in a cluster imports) we will have a table > in the database that contains what files was imported, from > what node, date and result (including a list of what resources > where imported, which was not, and stack trace if applicable). > The primary key will be the checksum of the file. We will also > add marker files (.imported or .failed). > The contents of the marker files will be a json object with > date imported, outcome (including stack trace if applicable) > as well as a complete list of what resources was successfully > imported, what where not. > > The files will also allow resolving system properties and > environment variables. For example: > > { > "secret": "${env.MYCLIENT_SECRET}" > } > > This will be very convenient for example with Docker as it > would be very easy to create a Docker image that extends ours > to add a few clients and users. > > It will also be convenient for examples as it will make it > possible to add the required clients and users to an existing > realm. > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151110/853909d8/attachment-0001.html From sthorger at redhat.com Wed Nov 11 03:01:38 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 11 Nov 2015 09:01:38 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: <5642093F.6030308@redhat.com> References: <5642093F.6030308@redhat.com> Message-ID: On 10 November 2015 at 16:11, Marek Posolda wrote: > On 09/11/15 14:09, Stian Thorgersen wrote: > > > > On 9 November 2015 at 13:35, Sebastien Blanc < > sblanc at redhat.com> wrote: > >> That would be really nice indeed ! >> But are the markers files not enough, instead of also having a table in >> the DB ? >> > > We need a way to prevent multiple nodes in a cluster to import the same > file. For example on Kerberos you end up spinning up multiple instances of > the same Docker image. > > I bet you meant 'Kubernetes' :-) > Yup > > > +1 for the improvements. Besides those I think that earlier or later, we > will need to solve long-running export+import where you want to import > 100.000 users. > +1 > > As I mentioned in another mail few weeks ago, we can have: > > 1) Table with the progress (51.000 users already imported, around 49.000 > remaining etc.) > We would still need to split into multiple files in either case. Having a single json file with 100K users is probably not going to perform very well. So what I proposed would actually work for long-running import as well. If each file has a manageable amount of users (say ~5 min to import) then each file will be marked as imported or failed. At least for now I don't think we should do smaller batches than one file. As long as one file is imported within the same TX then it's an all or nothing import. > 2) Concurrency and dividing the work among cluster nodes (Node1 will > import 50.000 users and node2 another 50.000 users) > This would be solved as well. Each node picks up a file that's not processed yet. Marks it in the DB and then gets to process it. > 3) Failover (Import won't be completely broken if cluster node crashes > after import 90.000, but can continue on other cluster nodes) > > I think the stuff I did recently for pre-loading offline sessions at > startup could be reused for this stuff too and it can handle (2) and (3) . > Also it can handle parallel import triggered from more cluster nodes. > > For example: currently if you trigger kubernetes with 2 cluster nodes, > both nodes will start to import same file at the same time because import > triggered by node1 is not yet finished before node2 is started, so there is > not yet existing DB record that file is already imported. With the stuff I > did, just the coordinator (node1) will start the import . Node2 will wait > until import triggered by node1 is finished, but at the same time it can > "help" to import some users (pages) if coordinator asks him to do so. This > impl is based on infinispan distributed executor service > http://infinispan.org/docs/5.3.x/user_guide/user_guide.html#_distributed_execution_framework > . > The DB record needs to be created before a node tries to import it, including a timestamp when it started the import. It should then be updated once the import is completed, with the result. Using the distributed execution framework sounds like a good idea though. How do you prevent scheduling the same job multiple times? For example if all nodes on startup scan the import folder and simply import everything they find, then there will be multiple of the same job. Not really a big deal as the first thing the job should do is check if there's a record in the DB already. > > > Marek > > > > >> >> >> On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen < >> sthorger at redhat.com> wrote: >> >>> Currently we support importing a complete realm definition using the >>> import/export feature. Issues with the current approach is: >>> >>> * Only complete realm - not possible to add to an existing realm >>> * No good feedback if import was successful or not >>> * Use of system properties to initiate the import is not very user >>> friendly >>> * Not very elegant for provisioning. For example a Docker image that >>> want's to bundle some initial setup ends up always running the import of a >>> realm, which is skipped if realm exists >>> >>> To solve this I've come up with the following proposal: >>> >>> Allow dropping representations to be imported into 'standalone/import'. >>> This should support creating a new realm as well as importing into an >>> existing realm. When importing into an existing realm we will have an >>> import strategy that is used to configure what happens if a resource exists >>> (user, role, identity provider, user federtation provider). The import >>> strategies are: >>> >>> * Skip - existing resources are skipped, >>> * Fail - if any resource exists nothing is imported >>> * Overwrite - any existing resources are deleted. >>> >>> The directory will be scanned at startup, but there will also be an >>> option to monitor this directory at runtime. >>> >>> To prevent a file being imported multiple times (also to make sure only >>> one node in a cluster imports) we will have a table in the database that >>> contains what files was imported, from what node, date and result >>> (including a list of what resources where imported, which was not, and >>> stack trace if applicable). The primary key will be the checksum of the >>> file. We will also add marker files (.imported or >> file>.failed). The contents of the marker files will be a json object with >>> date imported, outcome (including stack trace if applicable) as well as a >>> complete list of what resources was successfully imported, what where not. >>> >>> The files will also allow resolving system properties and environment >>> variables. For example: >>> >>> { >>> "secret": "${env.MYCLIENT_SECRET}" >>> } >>> >>> This will be very convenient for example with Docker as it would be very >>> easy to create a Docker image that extends ours to add a few clients and >>> users. >>> >>> It will also be convenient for examples as it will make it possible to >>> add the required clients and users to an existing realm. >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151111/2fafa1e3/attachment.html From sthorger at redhat.com Wed Nov 11 07:54:35 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 11 Nov 2015 13:54:35 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml In-Reply-To: <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> Message-ID: Would you be interested in contributing this feature? ATM we don't have anyone available that could work on it. A contribution would also need to include functional tests and documentation. If so I'm happy with going down the route of using the Hibernate specific classes. The remaining issue is figuring out how to deal with classloading. Looks like the following should work: * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider * JpaEntityProvider should have a single method "Class getEntities" * Implement org.hibernate.boot.registry.classloading.spi.ClassLoaderService - looks like this can just return null for everything except classForName where it would return the classes returned by the JpaEntityProvider implementations On 7 November 2015 at 23:39, Erik Mulder wrote: > *On 06/11/15 14:46, Stian Thorgersen wrote:* > *> We could use Hibernate directly to boostrap as long as it can return an > EntityManager. Do you know if that's possible?* > > I was a little quick to state that with Hibernate you can add extra entity > class names besides the one in persistence.xml, since I spotted a few > answers on StackOverflow that said it could be done. But they resolve > around classpath scanning or using a Spring managed Hibernate. Then I > thought: 'if Spring can do it, I can do it too' so I investigated the > Hibernate source code 'behind' > Persistence.createEntityManagerFactory(unitName, properties). After some > digging it turns out it's pretty simple to get extra class names in the > configuration. See code sample below. > > The only problem is that Hibernate will only find classes that are part of > the 'main' KeyCloak application, because of the way the Wildfly module > system and ClassLoader strategy work. The debugger showed me Hibernate has > these 3 class loaders available to look for classes: > 1. ModuleClassLoader for Module "deployment.keycloak-server.war:main" from > Service Module Loader > 2. ModuleClassLoader for Module "org.hibernate:main" from local module > loader > 3. sun.misc.Launcher$AppClassLoader > > Number 1 has all other KeyCloak modules in it, so the entity classes from > model-jpa will be found, but the wildfly-extensions module is missing, so > entities in classes in a jar in the providers folder cannot be found. Now > you guys obviously know a lot more about these internals, but as currently > configured, it seems to me there is no way to let Hibernate 'see' these > extra classes, since only the KeyCloak services module has a dependency on > wildfly-extensions. > > So I think it boils down to these decisions: > A. Do you accept a non-pure-JPA way of building the EntityManagerFactory > that has some ties to the Hibernate internals? > B. If A is no, than we're done. If yes, then you must find some way to get > the extra configured classes 'into' Hibernate. You could get the > wildfly-extensions module into scope of the Hibernate classloading. There > are serveral ways to configure Hibernate classloading or you could flip > some switches / dependencies in the module configuration. Another > alternative is to create a separate 'dropfolder' besides themes and > providers for JPA extensions, like 'models' or so and have that one be on > the Hibernate classpath. But I don't know the exact design principles > behind KeyCloak or the Wildfly module system. So maybe you have a better > solution or maybe you conclude that this is 'not done' in terms of the > architecture. > > Either way, I'd really appreciate some feedback on this and some thoughts > on whether this could be a possible addition to KeyCloak in your eyes. > > Thanks, Erik > > > Current JPA way. No way to 'interfere': > emf = Persistence.createEntityManagerFactory(unitName, properties); > > Alternative Hibernate only way with adding extra entity class names: > // Let Hibernate find and parse all 'persistence.xml' files found on the > classpath. > List persistenceUnits = > PersistenceXmlParser.locatePersistenceUnits(properties); > // Assume there is only one persistence unit found and that is the one we > need. This can be made more robust by checking on the persistence unit name. > ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = > persistenceUnits.get(0); > // Add extra class names. These could come from a 'JPA class name > provider' SPI or something alike. > > persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity", > "org.keycloak.models.jpa.entities.MerchantEntity"); > // Let Hibernate create an EntityManagerFactory out of the (enriched) > persistence unit configuration. > emf = Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, > properties).build(); > > > > > On 6 November 2015 at 14:29, Erik Mulder > wrote: > >> Thanks for pointing me explicitly to the SPI documentation. Of course >> that is exactly what I was looking for in my original question. I don't >> know how I overlooked this earlier! Probably I was not picking it up, >> because of almost a decade of developing on Spring projects, where this >> type of thing works differently. :-) >> >> I tried a quick test with a jar with one extra ProtocolMapper configured, >> put it in the providers folder and it worked like a charm! >> >> As for the JPA: We'll probably go with your suggestion of the separate >> EntityManagerFactory. Indeed there seems to be no way to 'programmatically >> extend' the list of entity classes in JPA besides editing or overwriting >> the persistence.xml. As you probably know it can be done in Hibernate, but >> I guess KeyCloak wants to stick to a generic JPA solution. That said, we >> might consider the Hibernate specific solution for our case, since being >> able to switch the JPA provider is not a requirement for us. And keeping >> the same connection/transaction is a lot easier in reasoning and debugging. >> > > We could use Hibernate directly to boostrap as long as it can return an > EntityManager. Do you know if that's possible? > > >> >> >> >> On 05/11/15 10:52, Stian Thorgersen wrote: >> >> The way to extend Keycloak is by implementing your own custom providers >> of the many SPIs we provide. Some SPIs are more stable (so marked as >> public) and others are not (so marked as private). If there are things that >> you want to customize that can't be done with an existing SPI then let us >> know and we may consider adding additional SPIs. >> >> On 4 November 2015 at 17:16, Erik Mulder < >> erik.mulder at docdatapayments.com> wrote: >> >>> Thanks for your response! >>> >>> Indeed we already did a proof of concept where we added a custom mapper >>> the way you described (didn't know it was 'protected' territory :). The >>> question is: do we have to override the file >>> 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper >>> in the original project or is there another way where we don't need to >>> touch the original sources and keep all our changes in a separate >>> project? And how can we do it such that it stays easy to upgrade to >>> newer KeyCloak releases? >>> >> >> Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look >> at the docs ( >> http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) >> and examples for other provider ( >> >> https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory >> ) >> >> >>> >>> As for JPA: it would be easier to integrate with the existing JPA >>> project. Again we are wondering whether to start modifying original >>> sources (like persistence.xml) or try to 'externalize' our changes >>> somehow and integrate them using existing 'hooks' in the system or maybe >>> merge projects during build. >>> >>> Maybe there is no good answer to this and we'll always be having some >>> manual merge pains when upgrading to new KeyCloak versions. We just >>> wanted to check if there are preferred ways to add functionality with >>> the least amount of impact on the original sources. >>> >> >> I initially wanted the ability to add custom entities to the >> JpaConnectProvider, but couldn't find a way to define entities >> programatically with JPA. To add your own persistence.xml you would have to >> define your own implementation of JpaConnectionProvider and change what is >> loaded by default (connectionsJpa/provider attribute in >> keycloak-server.json). >> >> Alternative, which is cleaner, but you end up with separate >> connection/transaction, is to create your own EntityManagerFactory. If it's >> only used by one provider (for example a custom UserFederationProvider) >> there's no need to add a connect provider (that's just a way to share one >> EntityManagerFactory between multiple providers) and you can just create it >> in the MyUserFederationProviderFactory. >> >> >>> >>> >>> On 04/11/15 15:30, Bill Burke wrote: >>> > Custom mappers should be possible. I didn't document it as I wasn't >>> > sure if we wanted to make the SPI public. Custom mappers should just >>> > follow the Provider SPI and they will be picked up. If you see the >>> > META-INF/services/... file in the resources directory of the "services" >>> > or "broker" modules you'll see how to set this up. >>> > >>> > As for extending the JPA datamodel, what you could do is write a new >>> JPA >>> > Connections Provider and plug that in. See connections/jpa. I'm not >>> > sure how you would handle the liquibase db migration. >>> > >>> > On 11/4/2015 6:03 AM, Erik Mulder wrote: >>> >> Hi everybody, >>> >> >>> >> Quick intro: I?m part of a development team in The Netherlands that is >>> >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >>> >> this and will use OpenID Connect to secure our REST services. It?s a >>> >> great product and seems to be the only one having both support for all >>> >> kinds of security standards and a model and GUI for users and roles. >>> >> Thanks for creating it! J >>> >> >>> >> (if this should be asked instead on the users mailing list, please >>> >> correct me and I?ll post it there) >>> >> >>> >> So far, so good, but we have some extra requirements that do not fit >>> >> into the base KeyCloak data model. See below for details if you?re >>> >> interested. My question is: what is the preferred way / best practice >>> to >>> >> extend the functionality of KeyCloak while keeping the impact on the >>> >> original sources to a minimum? Of course we could just fork the most >>> >> recent version and start hacking away, but we?d like to be able to >>> >> upgrade to newer versions of KeyCloak without too much hassle. >>> >> Possibilities that we?ve come up with so far: >>> >> >>> >> 1.Create completely separate modules that will extend the >>> functionality >>> >> the way we need. >>> >> >>> >> 2.Fork on Github, apply custom changes, and try to merge in updates >>> from >>> >> the master / release branches / tags >>> >> >>> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, >>> such >>> >> as Truezip >>> >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >>> >> manipulate zip files by adding/removing/replacing or Shade >>> >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine >>> multiple >>> >> jars to 1 'uber-jar' containing the contents of both and when >>> >> overlapping decide on conflicts through configuration. >>> >> >>> >> Of course number 1 is preferred, but I do not see how to add custom >>> >> mappers or JPA entities without making changes in the original module >>> >> files. The other options seem like valid alternatives, but maybe there >>> >> is better / standard way to do this. So any help / insight / shared >>> >> experience on this is much appreciated, thanks! >>> >> >>> >> Kind regards, >>> >> >>> >> Erik Mulder >>> >> >>> >> Senior Software Engineer >>> >> >>> >> Docdata Payments ? NL >>> >> >>> >> P.S. Details on why we want to extend the KeyCloak data model: (any >>> >> feedback on the contents of this P.S. is also welcome!) >>> >> >>> >> Our clients are merchants that have several webshops. We manage their >>> >> online payments (shopping cart checkout). We want to be able to let a >>> >> merchant manage their own users and let a user have different roles >>> for >>> >> different webshops within the same merchant. The overall possible >>> roles >>> >> are fixed though, no specific roles per merchant. We could create a >>> >> separate realm for every merchant, but then we need to duplicate all >>> >> roles every time. Furthermore, in KeyCloak there is no concept of a >>> role >>> >> within a certain context. This is very understandable, since every >>> >> situation has it?s own requirements. We did a proof of concept by >>> adding >>> >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >>> >> and adding a custom mapper that can put this information on the Access >>> >> token. Worked like a charm! But it does need some changes in the >>> >> KeyCloak modules and sources to work, hence the question above. >>> >> >>> >> >>> >> >>> >> _______________________________________________ >>> >> keycloak-dev mailing list >>> >> keycloak-dev at lists.jboss.org >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151111/c78e5bdf/attachment-0001.html From mposolda at redhat.com Wed Nov 11 09:23:48 2015 From: mposolda at redhat.com (Marek Posolda) Date: Wed, 11 Nov 2015 15:23:48 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: References: <5642093F.6030308@redhat.com> Message-ID: <56434F74.9030100@redhat.com> On 11/11/15 09:01, Stian Thorgersen wrote: > > > On 10 November 2015 at 16:11, Marek Posolda > wrote: > > On 09/11/15 14:09, Stian Thorgersen wrote: >> >> >> On 9 November 2015 at 13:35, Sebastien Blanc > > wrote: >> >> That would be really nice indeed ! >> But are the markers files not enough, instead of also having >> a table in the DB ? >> >> >> We need a way to prevent multiple nodes in a cluster to import >> the same file. For example on Kerberos you end up spinning up >> multiple instances of the same Docker image. > I bet you meant 'Kubernetes' :-) > > > Yup > > > > +1 for the improvements. Besides those I think that earlier or > later, we will need to solve long-running export+import where you > want to import 100.000 users. > > > +1 > > > As I mentioned in another mail few weeks ago, we can have: > > 1) Table with the progress (51.000 users already imported, around > 49.000 remaining etc.) > > > We would still need to split into multiple files in either case. > Having a single json file with 100K users is probably not going to > perform very well. So what I proposed would actually work for > long-running import as well. If each file has a manageable amount of > users (say ~5 min to import) then each file will be marked as imported > or failed. At least for now I don't think we should do smaller batches > than one file. As long as one file is imported within the same TX then > it's an all or nothing import. > > 2) Concurrency and dividing the work among cluster nodes (Node1 > will import 50.000 users and node2 another 50.000 users) > > > This would be solved as well. Each node picks up a file that's not > processed yet. Marks it in the DB and then gets to process it. > > 3) Failover (Import won't be completely broken if cluster node > crashes after import 90.000, but can continue on other cluster nodes) > > I think the stuff I did recently for pre-loading offline sessions > at startup could be reused for this stuff too and it can handle > (2) and (3) . Also it can handle parallel import triggered from > more cluster nodes. > > For example: currently if you trigger kubernetes with 2 cluster > nodes, both nodes will start to import same file at the same time > because import triggered by node1 is not yet finished before node2 > is started, so there is not yet existing DB record that file is > already imported. With the stuff I did, just the coordinator > (node1) will start the import . Node2 will wait until import > triggered by node1 is finished, but at the same time it can "help" > to import some users (pages) if coordinator asks him to do so. > This impl is based on infinispan distributed executor service > http://infinispan.org/docs/5.3.x/user_guide/user_guide.html#_distributed_execution_framework > . > > > The DB record needs to be created before a node tries to import it, > including a timestamp when it started the import. It should then be > updated once the import is completed, with the result. Using the > distributed execution framework sounds like a good idea though. How do > you prevent scheduling the same job multiple times? For example if all > nodes on startup scan the import folder and simply import everything > they find, then there will be multiple of the same job. Not really a > big deal as the first thing the job should do is check if there's a > record in the DB already. With distributed executor, it's the cluster coordinator, which coordinates which node would import what. It will send messages to cluster nodes like "Hey, please import the file testrealm-users-3.json with timestamp abcd123" . After node finishes the job, it notifies coordinator and coordinator will insert DB record and mark it as finished. So there is no DB record inserted before node starts import, because whole coordination is handled by the coordinator. Also there will never be same file imported more times by different cluster nodes. Only exception would be if cluster node crashes before import is finished. Then it needs to be reimported by other cluster node, but that's the case with DB locks as well. IMO the DB locks approach doesn't handle well crash of some cluster node. For example when node2 crashes unexpectedly when it's importing the file testrealm-users-3.json, the DB lock is held by this node, so other cluster nodes can't start on importing the file (until timeout occurs.) On the other hand, distributed executor approach may have issues if there is inconsistent content of the standalone/import directory among cluster nodes. However it can be solved, so that each node will need to send checksums of the files it has and coordinator will need to ensure that file with checksum "abcd123" is assigned just to the node which has this file. Marek > > > > Marek > > >> >> >> On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen >> > wrote: >> >> Currently we support importing a complete realm >> definition using the import/export feature. Issues with >> the current approach is: >> >> * Only complete realm - not possible to add to an >> existing realm >> * No good feedback if import was successful or not >> * Use of system properties to initiate the import is not >> very user friendly >> * Not very elegant for provisioning. For example a Docker >> image that want's to bundle some initial setup ends up >> always running the import of a realm, which is skipped if >> realm exists >> >> To solve this I've come up with the following proposal: >> >> Allow dropping representations to be imported into >> 'standalone/import'. This should support creating a new >> realm as well as importing into an existing realm. When >> importing into an existing realm we will have an import >> strategy that is used to configure what happens if a >> resource exists (user, role, identity provider, user >> federtation provider). The import strategies are: >> >> * Skip - existing resources are skipped, >> * Fail - if any resource exists nothing is imported >> * Overwrite - any existing resources are deleted. >> >> The directory will be scanned at startup, but there will >> also be an option to monitor this directory at runtime. >> >> To prevent a file being imported multiple times (also to >> make sure only one node in a cluster imports) we will >> have a table in the database that contains what files was >> imported, from what node, date and result (including a >> list of what resources where imported, which was not, and >> stack trace if applicable). The primary key will be the >> checksum of the file. We will also add marker files >> (.imported or .failed). The >> contents of the marker files will be a json object with >> date imported, outcome (including stack trace if >> applicable) as well as a complete list of what resources >> was successfully imported, what where not. >> >> The files will also allow resolving system properties and >> environment variables. For example: >> >> { >> "secret": "${env.MYCLIENT_SECRET}" >> } >> >> This will be very convenient for example with Docker as >> it would be very easy to create a Docker image that >> extends ours to add a few clients and users. >> >> It will also be convenient for examples as it will make >> it possible to add the required clients and users to an >> existing realm. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151111/0103247c/attachment-0001.html From sthorger at redhat.com Wed Nov 11 09:36:09 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 11 Nov 2015 15:36:09 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: <56434F74.9030100@redhat.com> References: <5642093F.6030308@redhat.com> <56434F74.9030100@redhat.com> Message-ID: On 11 November 2015 at 15:23, Marek Posolda wrote: > On 11/11/15 09:01, Stian Thorgersen wrote: > > > > On 10 November 2015 at 16:11, Marek Posolda wrote: > >> On 09/11/15 14:09, Stian Thorgersen wrote: >> >> >> >> On 9 November 2015 at 13:35, Sebastien Blanc < >> sblanc at redhat.com> wrote: >> >>> That would be really nice indeed ! >>> But are the markers files not enough, instead of also having a table in >>> the DB ? >>> >> >> We need a way to prevent multiple nodes in a cluster to import the same >> file. For example on Kerberos you end up spinning up multiple instances of >> the same Docker image. >> >> I bet you meant 'Kubernetes' :-) >> > > Yup > > >> >> >> +1 for the improvements. Besides those I think that earlier or later, we >> will need to solve long-running export+import where you want to import >> 100.000 users. >> > > +1 > > >> >> As I mentioned in another mail few weeks ago, we can have: >> >> 1) Table with the progress (51.000 users already imported, around 49.000 >> remaining etc.) >> > > We would still need to split into multiple files in either case. Having a > single json file with 100K users is probably not going to perform very > well. So what I proposed would actually work for long-running import as > well. If each file has a manageable amount of users (say ~5 min to import) > then each file will be marked as imported or failed. At least for now I > don't think we should do smaller batches than one file. As long as one file > is imported within the same TX then it's an all or nothing import. > > > >> 2) Concurrency and dividing the work among cluster nodes (Node1 will >> import 50.000 users and node2 another 50.000 users) >> > > This would be solved as well. Each node picks up a file that's not > processed yet. Marks it in the DB and then gets to process it. > > >> 3) Failover (Import won't be completely broken if cluster node crashes >> after import 90.000, but can continue on other cluster nodes) >> >> I think the stuff I did recently for pre-loading offline sessions at >> startup could be reused for this stuff too and it can handle (2) and (3) . >> Also it can handle parallel import triggered from more cluster nodes. >> >> For example: currently if you trigger kubernetes with 2 cluster nodes, >> both nodes will start to import same file at the same time because import >> triggered by node1 is not yet finished before node2 is started, so there is >> not yet existing DB record that file is already imported. With the stuff I >> did, just the coordinator (node1) will start the import . Node2 will wait >> until import triggered by node1 is finished, but at the same time it can >> "help" to import some users (pages) if coordinator asks him to do so. This >> impl is based on infinispan distributed executor service >> >> http://infinispan.org/docs/5.3.x/user_guide/user_guide.html#_distributed_execution_framework >> . >> > > The DB record needs to be created before a node tries to import it, > including a timestamp when it started the import. It should then be updated > once the import is completed, with the result. Using the distributed > execution framework sounds like a good idea though. How do you prevent > scheduling the same job multiple times? For example if all nodes on startup > scan the import folder and simply import everything they find, then there > will be multiple of the same job. Not really a big deal as the first thing > the job should do is check if there's a record in the DB already. > > With distributed executor, it's the cluster coordinator, which coordinates > which node would import what. It will send messages to cluster nodes like > "Hey, please import the file testrealm-users-3.json with timestamp abcd123" > . > > After node finishes the job, it notifies coordinator and coordinator will > insert DB record and mark it as finished. So there is no DB record inserted > before node starts import, because whole coordination is handled by the > coordinator. Also there will never be same file imported more times by > different cluster nodes. > > Only exception would be if cluster node crashes before import is finished. > Then it needs to be reimported by other cluster node, but that's the case > with DB locks as well. > > IMO the DB locks approach doesn't handle well crash of some cluster node. > For example when node2 crashes unexpectedly when it's importing the file > testrealm-users-3.json, the DB lock is held by this node, so other cluster > nodes can't start on importing the file (until timeout occurs.) > > On the other hand, distributed executor approach may have issues if there > is inconsistent content of the standalone/import directory among cluster > nodes. However it can be solved, so that each node will need to send > checksums of the files it has and coordinator will need to ensure that file > with checksum "abcd123" is assigned just to the node which has this file. > With Docker/Kubernetes all nodes would have the same files. At least initially. Would be nice if we could come up with a solution where you can just drop an additional file onto any node and have it imported. > > > Marek > > > >> >> >> Marek >> >> >> >> >>> >>> >>> On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen < >>> sthorger at redhat.com> wrote: >>> >>>> Currently we support importing a complete realm definition using the >>>> import/export feature. Issues with the current approach is: >>>> >>>> * Only complete realm - not possible to add to an existing realm >>>> * No good feedback if import was successful or not >>>> * Use of system properties to initiate the import is not very user >>>> friendly >>>> * Not very elegant for provisioning. For example a Docker image that >>>> want's to bundle some initial setup ends up always running the import of a >>>> realm, which is skipped if realm exists >>>> >>>> To solve this I've come up with the following proposal: >>>> >>>> Allow dropping representations to be imported into 'standalone/import'. >>>> This should support creating a new realm as well as importing into an >>>> existing realm. When importing into an existing realm we will have an >>>> import strategy that is used to configure what happens if a resource exists >>>> (user, role, identity provider, user federtation provider). The import >>>> strategies are: >>>> >>>> * Skip - existing resources are skipped, >>>> * Fail - if any resource exists nothing is imported >>>> * Overwrite - any existing resources are deleted. >>>> >>>> The directory will be scanned at startup, but there will also be an >>>> option to monitor this directory at runtime. >>>> >>>> To prevent a file being imported multiple times (also to make sure only >>>> one node in a cluster imports) we will have a table in the database that >>>> contains what files was imported, from what node, date and result >>>> (including a list of what resources where imported, which was not, and >>>> stack trace if applicable). The primary key will be the checksum of the >>>> file. We will also add marker files (.imported or >>> file>.failed). The contents of the marker files will be a json object with >>>> date imported, outcome (including stack trace if applicable) as well as a >>>> complete list of what resources was successfully imported, what where not. >>>> >>>> The files will also allow resolving system properties and environment >>>> variables. For example: >>>> >>>> { >>>> "secret": "${env.MYCLIENT_SECRET}" >>>> } >>>> >>>> This will be very convenient for example with Docker as it would be >>>> very easy to create a Docker image that extends ours to add a few clients >>>> and users. >>>> >>>> It will also be convenient for examples as it will make it possible to >>>> add the required clients and users to an existing realm. >>>> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>> >>> >> >> >> _______________________________________________ >> keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151111/7c8b9a88/attachment-0001.html From mposolda at redhat.com Wed Nov 11 09:51:12 2015 From: mposolda at redhat.com (Marek Posolda) Date: Wed, 11 Nov 2015 15:51:12 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: References: <5642093F.6030308@redhat.com> <56434F74.9030100@redhat.com> Message-ID: <564355E0.1040609@redhat.com> On 11/11/15 15:36, Stian Thorgersen wrote: > > > On 11 November 2015 at 15:23, Marek Posolda > wrote: > > On 11/11/15 09:01, Stian Thorgersen wrote: >> >> >> On 10 November 2015 at 16:11, Marek Posolda > > wrote: >> >> On 09/11/15 14:09, Stian Thorgersen wrote: >>> >>> >>> On 9 November 2015 at 13:35, Sebastien Blanc >>> > wrote: >>> >>> That would be really nice indeed ! >>> But are the markers files not enough, instead of also >>> having a table in the DB ? >>> >>> >>> We need a way to prevent multiple nodes in a cluster to >>> import the same file. For example on Kerberos you end up >>> spinning up multiple instances of the same Docker image. >> I bet you meant 'Kubernetes' :-) >> >> >> Yup >> >> >> >> +1 for the improvements. Besides those I think that earlier >> or later, we will need to solve long-running export+import >> where you want to import 100.000 users. >> >> >> +1 >> >> >> As I mentioned in another mail few weeks ago, we can have: >> >> 1) Table with the progress (51.000 users already imported, >> around 49.000 remaining etc.) >> >> >> We would still need to split into multiple files in either case. >> Having a single json file with 100K users is probably not going >> to perform very well. So what I proposed would actually work for >> long-running import as well. If each file has a manageable amount >> of users (say ~5 min to import) then each file will be marked as >> imported or failed. At least for now I don't think we should do >> smaller batches than one file. As long as one file is imported >> within the same TX then it's an all or nothing import. >> >> 2) Concurrency and dividing the work among cluster nodes >> (Node1 will import 50.000 users and node2 another 50.000 users) >> >> >> This would be solved as well. Each node picks up a file that's >> not processed yet. Marks it in the DB and then gets to process it. >> >> 3) Failover (Import won't be completely broken if cluster >> node crashes after import 90.000, but can continue on other >> cluster nodes) >> >> I think the stuff I did recently for pre-loading offline >> sessions at startup could be reused for this stuff too and it >> can handle (2) and (3) . Also it can handle parallel import >> triggered from more cluster nodes. >> >> For example: currently if you trigger kubernetes with 2 >> cluster nodes, both nodes will start to import same file at >> the same time because import triggered by node1 is not yet >> finished before node2 is started, so there is not yet >> existing DB record that file is already imported. With the >> stuff I did, just the coordinator (node1) will start the >> import . Node2 will wait until import triggered by node1 is >> finished, but at the same time it can "help" to import some >> users (pages) if coordinator asks him to do so. This impl is >> based on infinispan distributed executor service >> http://infinispan.org/docs/5.3.x/user_guide/user_guide.html#_distributed_execution_framework >> . >> >> >> The DB record needs to be created before a node tries to import >> it, including a timestamp when it started the import. It should >> then be updated once the import is completed, with the result. >> Using the distributed execution framework sounds like a good idea >> though. How do you prevent scheduling the same job multiple >> times? For example if all nodes on startup scan the import folder >> and simply import everything they find, then there will be >> multiple of the same job. Not really a big deal as the first >> thing the job should do is check if there's a record in the DB >> already. > With distributed executor, it's the cluster coordinator, which > coordinates which node would import what. It will send messages to > cluster nodes like "Hey, please import the file > testrealm-users-3.json with timestamp abcd123" . > > After node finishes the job, it notifies coordinator and > coordinator will insert DB record and mark it as finished. So > there is no DB record inserted before node starts import, because > whole coordination is handled by the coordinator. Also there will > never be same file imported more times by different cluster nodes. > > Only exception would be if cluster node crashes before import is > finished. Then it needs to be reimported by other cluster node, > but that's the case with DB locks as well. > > IMO the DB locks approach doesn't handle well crash of some > cluster node. For example when node2 crashes unexpectedly when > it's importing the file testrealm-users-3.json, the DB lock is > held by this node, so other cluster nodes can't start on importing > the file (until timeout occurs.) > > On the other hand, distributed executor approach may have issues > if there is inconsistent content of the standalone/import > directory among cluster nodes. However it can be solved, so that > each node will need to send checksums of the files it has and > coordinator will need to ensure that file with checksum "abcd123" > is assigned just to the node which has this file. > > > With Docker/Kubernetes all nodes would have the same files. At least > initially. Would be nice if we could come up with a solution where you > can just drop an additional file onto any node and have it imported. Exactly, was thinking about Docker too. Here we don't have any issue at all. The main question here is, do we want to support the scenario when various cluster nodes have different content? As I mentioned, distributed coordinator can handle it, so that each cluster node will send the checksums of the files it has and coordinator will always assign to node just the checksums, which it has. However regardless of distributed executor approach or DB locks approach, there may be still the issues. For example: 1) The file testrealm.json with checksum "abc" is triggered for import on node1 2) At the same time, admin will do some minor change in this file on node2 and save it. This will mean that checksum of the file on node2 will be changed to "def" 3) Node2 will trigger import of that file. So we have both node1 and node2 importing same file concurrently because the previously retrieved lock was for "abc" checksum, but now checksum is "def" This problem will be with both DB lock and DistributedExecutor approaches though... Marek > > > > Marek > >> >> >> Marek >> >> >>> >>> >>> On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen >>> > wrote: >>> >>> Currently we support importing a complete realm >>> definition using the import/export feature. Issues >>> with the current approach is: >>> >>> * Only complete realm - not possible to add to an >>> existing realm >>> * No good feedback if import was successful or not >>> * Use of system properties to initiate the import is >>> not very user friendly >>> * Not very elegant for provisioning. For example a >>> Docker image that want's to bundle some initial >>> setup ends up always running the import of a realm, >>> which is skipped if realm exists >>> >>> To solve this I've come up with the following proposal: >>> >>> Allow dropping representations to be imported into >>> 'standalone/import'. This should support creating a >>> new realm as well as importing into an existing >>> realm. When importing into an existing realm we will >>> have an import strategy that is used to configure >>> what happens if a resource exists (user, role, >>> identity provider, user federtation provider). The >>> import strategies are: >>> >>> * Skip - existing resources are skipped, >>> * Fail - if any resource exists nothing is imported >>> * Overwrite - any existing resources are deleted. >>> >>> The directory will be scanned at startup, but there >>> will also be an option to monitor this directory at >>> runtime. >>> >>> To prevent a file being imported multiple times >>> (also to make sure only one node in a cluster >>> imports) we will have a table in the database that >>> contains what files was imported, from what node, >>> date and result (including a list of what resources >>> where imported, which was not, and stack trace if >>> applicable). The primary key will be the checksum of >>> the file. We will also add marker files (>> file>.imported or .failed). The contents >>> of the marker files will be a json object with date >>> imported, outcome (including stack trace if >>> applicable) as well as a complete list of what >>> resources was successfully imported, what where not. >>> >>> The files will also allow resolving system >>> properties and environment variables. For example: >>> >>> { >>> "secret": "${env.MYCLIENT_SECRET}" >>> } >>> >>> This will be very convenient for example with Docker >>> as it would be very easy to create a Docker image >>> that extends ours to add a few clients and users. >>> >>> It will also be convenient for examples as it will >>> make it possible to add the required clients and >>> users to an existing realm. >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151111/3e0fc865/attachment-0001.html From mposolda at redhat.com Wed Nov 11 09:53:42 2015 From: mposolda at redhat.com (Marek Posolda) Date: Wed, 11 Nov 2015 15:53:42 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: <564355E0.1040609@redhat.com> References: <5642093F.6030308@redhat.com> <56434F74.9030100@redhat.com> <564355E0.1040609@redhat.com> Message-ID: <56435676.2050908@redhat.com> On 11/11/15 15:51, Marek Posolda wrote: > On 11/11/15 15:36, Stian Thorgersen wrote: >> >> >> On 11 November 2015 at 15:23, Marek Posolda > > wrote: >> >> On 11/11/15 09:01, Stian Thorgersen wrote: >>> >>> >>> On 10 November 2015 at 16:11, Marek Posolda >>> wrote: >>> >>> On 09/11/15 14:09, Stian Thorgersen wrote: >>>> >>>> >>>> On 9 November 2015 at 13:35, Sebastien Blanc >>>> wrote: >>>> >>>> That would be really nice indeed ! >>>> But are the markers files not enough, instead of also >>>> having a table in the DB ? >>>> >>>> >>>> We need a way to prevent multiple nodes in a cluster to >>>> import the same file. For example on Kerberos you end up >>>> spinning up multiple instances of the same Docker image. >>> I bet you meant 'Kubernetes' :-) >>> >>> >>> Yup >>> >>> >>> >>> +1 for the improvements. Besides those I think that earlier >>> or later, we will need to solve long-running export+import >>> where you want to import 100.000 users. >>> >>> >>> +1 >>> >>> >>> As I mentioned in another mail few weeks ago, we can have: >>> >>> 1) Table with the progress (51.000 users already imported, >>> around 49.000 remaining etc.) >>> >>> >>> We would still need to split into multiple files in either case. >>> Having a single json file with 100K users is probably not going >>> to perform very well. So what I proposed would actually work for >>> long-running import as well. If each file has a manageable >>> amount of users (say ~5 min to import) then each file will be >>> marked as imported or failed. At least for now I don't think we >>> should do smaller batches than one file. As long as one file is >>> imported within the same TX then it's an all or nothing import. >>> >>> 2) Concurrency and dividing the work among cluster nodes >>> (Node1 will import 50.000 users and node2 another 50.000 users) >>> >>> >>> This would be solved as well. Each node picks up a file that's >>> not processed yet. Marks it in the DB and then gets to process it. >>> >>> 3) Failover (Import won't be completely broken if cluster >>> node crashes after import 90.000, but can continue on other >>> cluster nodes) >>> >>> I think the stuff I did recently for pre-loading offline >>> sessions at startup could be reused for this stuff too and >>> it can handle (2) and (3) . Also it can handle parallel >>> import triggered from more cluster nodes. >>> >>> For example: currently if you trigger kubernetes with 2 >>> cluster nodes, both nodes will start to import same file at >>> the same time because import triggered by node1 is not yet >>> finished before node2 is started, so there is not yet >>> existing DB record that file is already imported. With the >>> stuff I did, just the coordinator (node1) will start the >>> import . Node2 will wait until import triggered by node1 is >>> finished, but at the same time it can "help" to import some >>> users (pages) if coordinator asks him to do so. This impl is >>> based on infinispan distributed executor service >>> http://infinispan.org/docs/5.3.x/user_guide/user_guide.html#_distributed_execution_framework >>> . >>> >>> >>> The DB record needs to be created before a node tries to import >>> it, including a timestamp when it started the import. It should >>> then be updated once the import is completed, with the result. >>> Using the distributed execution framework sounds like a good >>> idea though. How do you prevent scheduling the same job multiple >>> times? For example if all nodes on startup scan the import >>> folder and simply import everything they find, then there will >>> be multiple of the same job. Not really a big deal as the first >>> thing the job should do is check if there's a record in the DB >>> already. >> With distributed executor, it's the cluster coordinator, which >> coordinates which node would import what. It will send messages >> to cluster nodes like "Hey, please import the file >> testrealm-users-3.json with timestamp abcd123" . >> >> After node finishes the job, it notifies coordinator and >> coordinator will insert DB record and mark it as finished. So >> there is no DB record inserted before node starts import, because >> whole coordination is handled by the coordinator. Also there will >> never be same file imported more times by different cluster nodes. >> >> Only exception would be if cluster node crashes before import is >> finished. Then it needs to be reimported by other cluster node, >> but that's the case with DB locks as well. >> >> IMO the DB locks approach doesn't handle well crash of some >> cluster node. For example when node2 crashes unexpectedly when >> it's importing the file testrealm-users-3.json, the DB lock is >> held by this node, so other cluster nodes can't start on >> importing the file (until timeout occurs.) >> >> On the other hand, distributed executor approach may have issues >> if there is inconsistent content of the standalone/import >> directory among cluster nodes. However it can be solved, so that >> each node will need to send checksums of the files it has and >> coordinator will need to ensure that file with checksum "abcd123" >> is assigned just to the node which has this file. >> >> >> With Docker/Kubernetes all nodes would have the same files. At least >> initially. Would be nice if we could come up with a solution where >> you can just drop an additional file onto any node and have it imported. > Exactly, was thinking about Docker too. Here we don't have any issue > at all. > > The main question here is, do we want to support the scenario when > various cluster nodes have different content? As I mentioned, > distributed coordinator can handle it, so that each cluster node will > send the checksums of the files it has and coordinator will always > assign to node just the checksums, which it has. > > However regardless of distributed executor approach or DB locks > approach, there may be still the issues. For example: > 1) The file testrealm.json with checksum "abc" is triggered for import > on node1 > 2) At the same time, admin will do some minor change in this file on > node2 and save it. This will mean that checksum of the file on node2 > will be changed to "def" > 3) Node2 will trigger import of that file. So we have both node1 and > node2 importing same file concurrently because the previously > retrieved lock was for "abc" checksum, but now checksum is "def" > > This problem will be with both DB lock and DistributedExecutor > approaches though... Possible solution for this issue is, that when import is already in progress, the newly added or changed checksums will be ignored. The checksums will be always checked just at start of the import. Marek > > Marek >> >> >> >> Marek >> >>> >>> >>> Marek >>> >>> >>>> >>>> >>>> On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen >>>> wrote: >>>> >>>> Currently we support importing a complete realm >>>> definition using the import/export feature. Issues >>>> with the current approach is: >>>> >>>> * Only complete realm - not possible to add to an >>>> existing realm >>>> * No good feedback if import was successful or not >>>> * Use of system properties to initiate the import >>>> is not very user friendly >>>> * Not very elegant for provisioning. For example a >>>> Docker image that want's to bundle some initial >>>> setup ends up always running the import of a realm, >>>> which is skipped if realm exists >>>> >>>> To solve this I've come up with the following proposal: >>>> >>>> Allow dropping representations to be imported into >>>> 'standalone/import'. This should support creating a >>>> new realm as well as importing into an existing >>>> realm. When importing into an existing realm we >>>> will have an import strategy that is used to >>>> configure what happens if a resource exists (user, >>>> role, identity provider, user federtation >>>> provider). The import strategies are: >>>> >>>> * Skip - existing resources are skipped, >>>> * Fail - if any resource exists nothing is imported >>>> * Overwrite - any existing resources are deleted. >>>> >>>> The directory will be scanned at startup, but there >>>> will also be an option to monitor this directory at >>>> runtime. >>>> >>>> To prevent a file being imported multiple times >>>> (also to make sure only one node in a cluster >>>> imports) we will have a table in the database that >>>> contains what files was imported, from what node, >>>> date and result (including a list of what resources >>>> where imported, which was not, and stack trace if >>>> applicable). The primary key will be the checksum >>>> of the file. We will also add marker files (>>> file>.imported or .failed). The contents >>>> of the marker files will be a json object with date >>>> imported, outcome (including stack trace if >>>> applicable) as well as a complete list of what >>>> resources was successfully imported, what where not. >>>> >>>> The files will also allow resolving system >>>> properties and environment variables. For example: >>>> >>>> { >>>> "secret": "${env.MYCLIENT_SECRET}" >>>> } >>>> >>>> This will be very convenient for example with >>>> Docker as it would be very easy to create a Docker >>>> image that extends ours to add a few clients and users. >>>> >>>> It will also be convenient for examples as it will >>>> make it possible to add the required clients and >>>> users to an existing realm. >>>> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >> >> > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151111/4e41650f/attachment-0001.html From sthorger at redhat.com Wed Nov 11 09:54:54 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 11 Nov 2015 15:54:54 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: <564355E0.1040609@redhat.com> References: <5642093F.6030308@redhat.com> <56434F74.9030100@redhat.com> <564355E0.1040609@redhat.com> Message-ID: On 11 November 2015 at 15:51, Marek Posolda wrote: > On 11/11/15 15:36, Stian Thorgersen wrote: > > > > On 11 November 2015 at 15:23, Marek Posolda wrote: > >> On 11/11/15 09:01, Stian Thorgersen wrote: >> >> >> >> On 10 November 2015 at 16:11, Marek Posolda < >> mposolda at redhat.com> wrote: >> >>> On 09/11/15 14:09, Stian Thorgersen wrote: >>> >>> >>> >>> On 9 November 2015 at 13:35, Sebastien Blanc < >>> sblanc at redhat.com> wrote: >>> >>>> That would be really nice indeed ! >>>> But are the markers files not enough, instead of also having a table in >>>> the DB ? >>>> >>> >>> We need a way to prevent multiple nodes in a cluster to import the same >>> file. For example on Kerberos you end up spinning up multiple instances of >>> the same Docker image. >>> >>> I bet you meant 'Kubernetes' :-) >>> >> >> Yup >> >> >>> >>> >>> +1 for the improvements. Besides those I think that earlier or later, we >>> will need to solve long-running export+import where you want to import >>> 100.000 users. >>> >> >> +1 >> >> >>> >>> As I mentioned in another mail few weeks ago, we can have: >>> >>> 1) Table with the progress (51.000 users already imported, around 49.000 >>> remaining etc.) >>> >> >> We would still need to split into multiple files in either case. Having a >> single json file with 100K users is probably not going to perform very >> well. So what I proposed would actually work for long-running import as >> well. If each file has a manageable amount of users (say ~5 min to import) >> then each file will be marked as imported or failed. At least for now I >> don't think we should do smaller batches than one file. As long as one file >> is imported within the same TX then it's an all or nothing import. >> >> >> >>> 2) Concurrency and dividing the work among cluster nodes (Node1 will >>> import 50.000 users and node2 another 50.000 users) >>> >> >> This would be solved as well. Each node picks up a file that's not >> processed yet. Marks it in the DB and then gets to process it. >> >> >>> 3) Failover (Import won't be completely broken if cluster node crashes >>> after import 90.000, but can continue on other cluster nodes) >>> >>> I think the stuff I did recently for pre-loading offline sessions at >>> startup could be reused for this stuff too and it can handle (2) and (3) . >>> Also it can handle parallel import triggered from more cluster nodes. >>> >>> For example: currently if you trigger kubernetes with 2 cluster nodes, >>> both nodes will start to import same file at the same time because import >>> triggered by node1 is not yet finished before node2 is started, so there is >>> not yet existing DB record that file is already imported. With the stuff I >>> did, just the coordinator (node1) will start the import . Node2 will wait >>> until import triggered by node1 is finished, but at the same time it can >>> "help" to import some users (pages) if coordinator asks him to do so. This >>> impl is based on infinispan distributed executor service >>> >>> http://infinispan.org/docs/5.3.x/user_guide/user_guide.html#_distributed_execution_framework >>> . >>> >> >> The DB record needs to be created before a node tries to import it, >> including a timestamp when it started the import. It should then be updated >> once the import is completed, with the result. Using the distributed >> execution framework sounds like a good idea though. How do you prevent >> scheduling the same job multiple times? For example if all nodes on startup >> scan the import folder and simply import everything they find, then there >> will be multiple of the same job. Not really a big deal as the first thing >> the job should do is check if there's a record in the DB already. >> >> With distributed executor, it's the cluster coordinator, which >> coordinates which node would import what. It will send messages to cluster >> nodes like "Hey, please import the file testrealm-users-3.json with >> timestamp abcd123" . >> >> After node finishes the job, it notifies coordinator and coordinator will >> insert DB record and mark it as finished. So there is no DB record inserted >> before node starts import, because whole coordination is handled by the >> coordinator. Also there will never be same file imported more times by >> different cluster nodes. >> >> Only exception would be if cluster node crashes before import is >> finished. Then it needs to be reimported by other cluster node, but that's >> the case with DB locks as well. >> >> IMO the DB locks approach doesn't handle well crash of some cluster node. >> For example when node2 crashes unexpectedly when it's importing the file >> testrealm-users-3.json, the DB lock is held by this node, so other cluster >> nodes can't start on importing the file (until timeout occurs.) >> >> On the other hand, distributed executor approach may have issues if there >> is inconsistent content of the standalone/import directory among cluster >> nodes. However it can be solved, so that each node will need to send >> checksums of the files it has and coordinator will need to ensure that file >> with checksum "abcd123" is assigned just to the node which has this file. >> > > With Docker/Kubernetes all nodes would have the same files. At least > initially. Would be nice if we could come up with a solution where you can > just drop an additional file onto any node and have it imported. > > Exactly, was thinking about Docker too. Here we don't have any issue at > all. > > The main question here is, do we want to support the scenario when various > cluster nodes have different content? As I mentioned, distributed > coordinator can handle it, so that each cluster node will send the > checksums of the files it has and coordinator will always assign to node > just the checksums, which it has. > That would be a nice addition IMO > > However regardless of distributed executor approach or DB locks approach, > there may be still the issues. For example: > 1) The file testrealm.json with checksum "abc" is triggered for import on > node1 > 2) At the same time, admin will do some minor change in this file on node2 > and save it. This will mean that checksum of the file on node2 will be > changed to "def" > 3) Node2 will trigger import of that file. So we have both node1 and node2 > importing same file concurrently because the previously retrieved lock was > for "abc" checksum, but now checksum is "def" > > This problem will be with both DB lock and DistributedExecutor approaches > though... > Maybe a better approach is to elect a single node that can perform imports and only allow one import at the time? > > > Marek > > > >> >> >> Marek >> >> >> >>> >>> >>> Marek >>> >>> >>> >>> >>>> >>>> >>>> On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen < >>>> sthorger at redhat.com> wrote: >>>> >>>>> Currently we support importing a complete realm definition using the >>>>> import/export feature. Issues with the current approach is: >>>>> >>>>> * Only complete realm - not possible to add to an existing realm >>>>> * No good feedback if import was successful or not >>>>> * Use of system properties to initiate the import is not very user >>>>> friendly >>>>> * Not very elegant for provisioning. For example a Docker image that >>>>> want's to bundle some initial setup ends up always running the import of a >>>>> realm, which is skipped if realm exists >>>>> >>>>> To solve this I've come up with the following proposal: >>>>> >>>>> Allow dropping representations to be imported into >>>>> 'standalone/import'. This should support creating a new realm as well as >>>>> importing into an existing realm. When importing into an existing realm we >>>>> will have an import strategy that is used to configure what happens if a >>>>> resource exists (user, role, identity provider, user federtation provider). >>>>> The import strategies are: >>>>> >>>>> * Skip - existing resources are skipped, >>>>> * Fail - if any resource exists nothing is imported >>>>> * Overwrite - any existing resources are deleted. >>>>> >>>>> The directory will be scanned at startup, but there will also be an >>>>> option to monitor this directory at runtime. >>>>> >>>>> To prevent a file being imported multiple times (also to make sure >>>>> only one node in a cluster imports) we will have a table in the database >>>>> that contains what files was imported, from what node, date and result >>>>> (including a list of what resources where imported, which was not, and >>>>> stack trace if applicable). The primary key will be the checksum of the >>>>> file. We will also add marker files (.imported or >>>> file>.failed). The contents of the marker files will be a json object with >>>>> date imported, outcome (including stack trace if applicable) as well as a >>>>> complete list of what resources was successfully imported, what where not. >>>>> >>>>> The files will also allow resolving system properties and environment >>>>> variables. For example: >>>>> >>>>> { >>>>> "secret": "${env.MYCLIENT_SECRET}" >>>>> } >>>>> >>>>> This will be very convenient for example with Docker as it would be >>>>> very easy to create a Docker image that extends ours to add a few clients >>>>> and users. >>>>> >>>>> It will also be convenient for examples as it will make it possible to >>>>> add the required clients and users to an existing realm. >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>> >>>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151111/720ba427/attachment-0001.html From sthorger at redhat.com Wed Nov 11 10:01:36 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 11 Nov 2015 16:01:36 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: <56435676.2050908@redhat.com> References: <5642093F.6030308@redhat.com> <56434F74.9030100@redhat.com> <564355E0.1040609@redhat.com> <56435676.2050908@redhat.com> Message-ID: What about if we remove files as they are being imported? Something like: * When we detect a new file on a node, we send the name + checksum to the cordinator * The cordinator then checks if this file has already been imported * If it's imported it sends out a message stating that file with name + checksum is already imported and all nodes delete this file * If it's not imported it picks one node that is responsible to import the file (that would be the first node that sends the message about the file+checksum). This node will rename the file to .importing * Once the file has been imported it's renamed to .imported or .failed On 11 November 2015 at 15:53, Marek Posolda wrote: > On 11/11/15 15:51, Marek Posolda wrote: > > On 11/11/15 15:36, Stian Thorgersen wrote: > > > > On 11 November 2015 at 15:23, Marek Posolda wrote: > >> On 11/11/15 09:01, Stian Thorgersen wrote: >> >> >> >> On 10 November 2015 at 16:11, Marek Posolda < >> mposolda at redhat.com> wrote: >> >>> On 09/11/15 14:09, Stian Thorgersen wrote: >>> >>> >>> >>> On 9 November 2015 at 13:35, Sebastien Blanc < >>> sblanc at redhat.com> wrote: >>> >>>> That would be really nice indeed ! >>>> But are the markers files not enough, instead of also having a table in >>>> the DB ? >>>> >>> >>> We need a way to prevent multiple nodes in a cluster to import the same >>> file. For example on Kerberos you end up spinning up multiple instances of >>> the same Docker image. >>> >>> I bet you meant 'Kubernetes' :-) >>> >> >> Yup >> >> >>> >>> >>> +1 for the improvements. Besides those I think that earlier or later, we >>> will need to solve long-running export+import where you want to import >>> 100.000 users. >>> >> >> +1 >> >> >>> >>> As I mentioned in another mail few weeks ago, we can have: >>> >>> 1) Table with the progress (51.000 users already imported, around 49.000 >>> remaining etc.) >>> >> >> We would still need to split into multiple files in either case. Having a >> single json file with 100K users is probably not going to perform very >> well. So what I proposed would actually work for long-running import as >> well. If each file has a manageable amount of users (say ~5 min to import) >> then each file will be marked as imported or failed. At least for now I >> don't think we should do smaller batches than one file. As long as one file >> is imported within the same TX then it's an all or nothing import. >> >> >> >>> 2) Concurrency and dividing the work among cluster nodes (Node1 will >>> import 50.000 users and node2 another 50.000 users) >>> >> >> This would be solved as well. Each node picks up a file that's not >> processed yet. Marks it in the DB and then gets to process it. >> >> >>> 3) Failover (Import won't be completely broken if cluster node crashes >>> after import 90.000, but can continue on other cluster nodes) >>> >>> I think the stuff I did recently for pre-loading offline sessions at >>> startup could be reused for this stuff too and it can handle (2) and (3) . >>> Also it can handle parallel import triggered from more cluster nodes. >>> >>> For example: currently if you trigger kubernetes with 2 cluster nodes, >>> both nodes will start to import same file at the same time because import >>> triggered by node1 is not yet finished before node2 is started, so there is >>> not yet existing DB record that file is already imported. With the stuff I >>> did, just the coordinator (node1) will start the import . Node2 will wait >>> until import triggered by node1 is finished, but at the same time it can >>> "help" to import some users (pages) if coordinator asks him to do so. This >>> impl is based on infinispan distributed executor service >>> http://infinispan.org/docs/5.3.x/user_guide/user_guide.html#_distributed_execution_framework >>> . >>> >> >> The DB record needs to be created before a node tries to import it, >> including a timestamp when it started the import. It should then be updated >> once the import is completed, with the result. Using the distributed >> execution framework sounds like a good idea though. How do you prevent >> scheduling the same job multiple times? For example if all nodes on startup >> scan the import folder and simply import everything they find, then there >> will be multiple of the same job. Not really a big deal as the first thing >> the job should do is check if there's a record in the DB already. >> >> With distributed executor, it's the cluster coordinator, which >> coordinates which node would import what. It will send messages to cluster >> nodes like "Hey, please import the file testrealm-users-3.json with >> timestamp abcd123" . >> >> After node finishes the job, it notifies coordinator and coordinator will >> insert DB record and mark it as finished. So there is no DB record inserted >> before node starts import, because whole coordination is handled by the >> coordinator. Also there will never be same file imported more times by >> different cluster nodes. >> >> Only exception would be if cluster node crashes before import is >> finished. Then it needs to be reimported by other cluster node, but that's >> the case with DB locks as well. >> >> IMO the DB locks approach doesn't handle well crash of some cluster node. >> For example when node2 crashes unexpectedly when it's importing the file >> testrealm-users-3.json, the DB lock is held by this node, so other cluster >> nodes can't start on importing the file (until timeout occurs.) >> >> On the other hand, distributed executor approach may have issues if there >> is inconsistent content of the standalone/import directory among cluster >> nodes. However it can be solved, so that each node will need to send >> checksums of the files it has and coordinator will need to ensure that file >> with checksum "abcd123" is assigned just to the node which has this file. >> > > With Docker/Kubernetes all nodes would have the same files. At least > initially. Would be nice if we could come up with a solution where you can > just drop an additional file onto any node and have it imported. > > Exactly, was thinking about Docker too. Here we don't have any issue at > all. > > The main question here is, do we want to support the scenario when various > cluster nodes have different content? As I mentioned, distributed > coordinator can handle it, so that each cluster node will send the > checksums of the files it has and coordinator will always assign to node > just the checksums, which it has. > > However regardless of distributed executor approach or DB locks approach, > there may be still the issues. For example: > 1) The file testrealm.json with checksum "abc" is triggered for import on > node1 > 2) At the same time, admin will do some minor change in this file on node2 > and save it. This will mean that checksum of the file on node2 will be > changed to "def" > 3) Node2 will trigger import of that file. So we have both node1 and node2 > importing same file concurrently because the previously retrieved lock was > for "abc" checksum, but now checksum is "def" > > This problem will be with both DB lock and DistributedExecutor approaches > though... > > Possible solution for this issue is, that when import is already in > progress, the newly added or changed checksums will be ignored. The > checksums will be always checked just at start of the import. > > Marek > > > Marek > > > >> >> >> Marek >> >> >> >>> >>> >>> Marek >>> >>> >>> >>> >>>> >>>> >>>> On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen < >>>> sthorger at redhat.com> wrote: >>>> >>>>> Currently we support importing a complete realm definition using the >>>>> import/export feature. Issues with the current approach is: >>>>> >>>>> * Only complete realm - not possible to add to an existing realm >>>>> * No good feedback if import was successful or not >>>>> * Use of system properties to initiate the import is not very user >>>>> friendly >>>>> * Not very elegant for provisioning. For example a Docker image that >>>>> want's to bundle some initial setup ends up always running the import of a >>>>> realm, which is skipped if realm exists >>>>> >>>>> To solve this I've come up with the following proposal: >>>>> >>>>> Allow dropping representations to be imported into >>>>> 'standalone/import'. This should support creating a new realm as well as >>>>> importing into an existing realm. When importing into an existing realm we >>>>> will have an import strategy that is used to configure what happens if a >>>>> resource exists (user, role, identity provider, user federtation provider). >>>>> The import strategies are: >>>>> >>>>> * Skip - existing resources are skipped, >>>>> * Fail - if any resource exists nothing is imported >>>>> * Overwrite - any existing resources are deleted. >>>>> >>>>> The directory will be scanned at startup, but there will also be an >>>>> option to monitor this directory at runtime. >>>>> >>>>> To prevent a file being imported multiple times (also to make sure >>>>> only one node in a cluster imports) we will have a table in the database >>>>> that contains what files was imported, from what node, date and result >>>>> (including a list of what resources where imported, which was not, and >>>>> stack trace if applicable). The primary key will be the checksum of the >>>>> file. We will also add marker files (.imported or >>>> file>.failed). The contents of the marker files will be a json object with >>>>> date imported, outcome (including stack trace if applicable) as well as a >>>>> complete list of what resources was successfully imported, what where not. >>>>> >>>>> The files will also allow resolving system properties and environment >>>>> variables. For example: >>>>> >>>>> { >>>>> "secret": "${env.MYCLIENT_SECRET}" >>>>> } >>>>> >>>>> This will be very convenient for example with Docker as it would be >>>>> very easy to create a Docker image that extends ours to add a few clients >>>>> and users. >>>>> >>>>> It will also be convenient for examples as it will make it possible to >>>>> add the required clients and users to an existing realm. >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>> >>>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> >> >> > > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151111/8d008d9f/attachment-0001.html From matthew.casperson at autogeneral.com.au Wed Nov 11 17:05:37 2015 From: matthew.casperson at autogeneral.com.au (Matthew Casperson) Date: Thu, 12 Nov 2015 08:05:37 +1000 Subject: [keycloak-dev] Cluster security Message-ID: The docs state that "By default there's nothing to prevent unauthorized nodes from joining the cluster and sending potentially malicious messages to the cluster." ( http://keycloak.github.io/docs/userguide/keycloak-server/html/clustering.html ) Is this still the case if the jgroups stack in Wildfly has implemented the AUTH protocol? For example, the Openshift Wildfly config looks something like this: ${env.OPENSHIFT_GEAR_DNS} ${env.OPENSHIFT_WILDFLY_CLUSTER_PROXY_PORT} ${env.OPENSHIFT_WILDFLY_CLUSTER_PORT} ${env.OPENSHIFT_WILDFLY_IP} true 30000 ${env.OPENSHIFT_WILDFLY_CLUSTER} 0 1 org.jgroups.auth.MD5Token SHA ${env.OPENSHIFT_SECRET_TOKEN} -- *Matthew Casperson* *Senior Front End Developer* Technology, Space & Distribution Auto & General Holdings Pty Ltd P: 07) 3377 8751 (Direct: 3377 8751) F: 07) 3377 8833 -- This email is sent by Auto & General Insurance Company Ltd, Auto & General Services Pty Ltd, Auto & General Holdings Pty Ltd or a related body corporate (Auto & General) and is for the intended addressee. The views expressed in this email and attachments (email) reflect the views of the stated author but may not reflect views of Auto & General. This email is confidential and subject to copyright. It may be privileged. If you are not the intended addressee, confidentiality and privilege have not been waived and any use, interference with, or disclosure of this email is unauthorised. If you are not the intended addressee please immediately notify the sender and then delete the email. Auto & General does not warrant that this email is error or virus free. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/51a9c0f1/attachment.html From sthorger at redhat.com Thu Nov 12 02:52:28 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 12 Nov 2015 08:52:28 +0100 Subject: [keycloak-dev] Cluster security In-Reply-To: References: Message-ID: I believe enabling auth is sufficient, but I'm not 100% sure. I've sent an email to JGroups mailing list to confirm and I'll let you know. On 11 November 2015 at 23:05, Matthew Casperson < matthew.casperson at autogeneral.com.au> wrote: > The docs state that "By default there's nothing to prevent unauthorized > nodes from joining the cluster and sending potentially malicious messages > to the cluster." ( > http://keycloak.github.io/docs/userguide/keycloak-server/html/clustering.html > ) > > Is this still the case if the jgroups stack in Wildfly has implemented the > AUTH protocol? For example, the Openshift Wildfly config looks something > like this: > > > > ${env.OPENSHIFT_GEAR_DNS} > ${env.OPENSHIFT_WILDFLY_CLUSTER_PROXY_PORT} > ${env.OPENSHIFT_WILDFLY_CLUSTER_PORT} > ${env.OPENSHIFT_WILDFLY_IP} > true > > > 30000 > ${env.OPENSHIFT_WILDFLY_CLUSTER} > 0 > 1 > > > > > > > > > > org.jgroups.auth.MD5Token > SHA > ${env.OPENSHIFT_SECRET_TOKEN} > > > > > > > > > > > -- > *Matthew Casperson* > *Senior Front End Developer* > Technology, Space & Distribution > Auto & General Holdings Pty Ltd > P: 07) 3377 8751 (Direct: 3377 8751) > F: 07) 3377 8833 > > > > This email is sent by Auto & General Insurance Company Ltd, Auto & General Services Pty Ltd, Auto & General Holdings Pty Ltd or a related body corporate (Auto & General) and is for the intended addressee. > The views expressed in this email and attachments (email) reflect the views of the stated author but may not reflect views of Auto & General. This email is confidential and subject to copyright. > It may be privileged. If you are not the intended addressee, confidentiality and privilege have not been waived and any use, interference with, or disclosure of this email is unauthorised. > If you are not the intended addressee please immediately notify the sender and then delete the email. Auto & General does not warrant that this email is error or virus free. > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/fd938300/attachment-0001.html From mposolda at redhat.com Thu Nov 12 03:07:08 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 12 Nov 2015 09:07:08 +0100 Subject: [keycloak-dev] Import proposal In-Reply-To: References: <5642093F.6030308@redhat.com> <56434F74.9030100@redhat.com> <564355E0.1040609@redhat.com> <56435676.2050908@redhat.com> Message-ID: <564448AC.1090202@redhat.com> Deleting the files should be good. One tricky thing are also dependencies among files. We need to make sure that file "testrealm.json" with realm definition is imported before users file "testrealm-users-X.json" . Also master realm should be always imported first. We can solve this by divide files into groups: 1) File with master realm (if exists) 2) Other files with realm definitions 3) All user's files We need to ensure that import of each group will start after previous group is fully finished. But this is solvable too. So far I can see this as my favourite workflow: a) When import is triggered, coordinator will ask all cluster nodes for file checksums they have. Each node will check DB and delete the files, which are already in DB (file with that checksum was already imported in previous import iteration). Then node send the checksums (just those which were not deleted) and to which group each checksum belongs (master, realms, users) b) Import of group1 starts. Coordinator will elect the node, which has the file with particular checksum available. For example if checksum "abcd" is available on node1 and node3, any of those 2 nodes are elected. c) Once import of file "abcd" is successful, node will save the record "abcd - IMPORT SUCCESS" to DB and send message to coordinator about finished import. d) If node1 crashes, the import of "abcd" file is immediatelly re-triggered by coordinator on node3. Similarly if file with checksum "abcd" is not available on node1 (this can happen if admin edited the file on node1 and saved it. The checksum of the file will be changed in this case), the import is re-triggered on node3 too. e) Once group1 (master realm) is finished, go back to step (b) and start with group2 (other realms), then again with group3 (user's files) f) The checksums are collected just at startup (step a). If someone change the file on any node during import iteration, it will be new checksum and this one will be ignored until this import iteration is finished. g) If new node joins the cluster, it can start helping with import as long as he has the files with checksums collected at step (a). At the end of the import iteration, there are DB records of all successfully imported checksums. When new import iteration is triggered, the nodes will delete the files already imported (we can also do that at the end of the import iteration too) Maybe it sounds a bit complicated, but I can see this as most performant workflow and vulnerable to crashes of any cluster node. Also it's vulnerable to the situation when admin changed some file when import is in progress. Marek On 11/11/15 16:01, Stian Thorgersen wrote: > What about if we remove files as they are being imported? > > Something like: > > * When we detect a new file on a node, we send the name + checksum to > the cordinator > * The cordinator then checks if this file has already been imported > * If it's imported it sends out a message stating that file with name > + checksum is already imported and all nodes delete this file > * If it's not imported it picks one node that is responsible to import > the file (that would be the first node that sends the message about > the file+checksum). This node will rename the file to .importing > * Once the file has been imported it's renamed to .imported or .failed > > On 11 November 2015 at 15:53, Marek Posolda > wrote: > > On 11/11/15 15:51, Marek Posolda wrote: >> On 11/11/15 15:36, Stian Thorgersen wrote: >>> >>> >>> On 11 November 2015 at 15:23, Marek Posolda >> > wrote: >>> >>> On 11/11/15 09:01, Stian Thorgersen wrote: >>>> >>>> >>>> On 10 November 2015 at 16:11, Marek Posolda >>>> > wrote: >>>> >>>> On 09/11/15 14:09, Stian Thorgersen wrote: >>>>> >>>>> >>>>> On 9 November 2015 at 13:35, Sebastien Blanc >>>>> > wrote: >>>>> >>>>> That would be really nice indeed ! >>>>> But are the markers files not enough, instead of >>>>> also having a table in the DB ? >>>>> >>>>> >>>>> We need a way to prevent multiple nodes in a cluster >>>>> to import the same file. For example on Kerberos you >>>>> end up spinning up multiple instances of the same >>>>> Docker image. >>>> I bet you meant 'Kubernetes' :-) >>>> >>>> >>>> Yup >>>> >>>> >>>> >>>> +1 for the improvements. Besides those I think that >>>> earlier or later, we will need to solve long-running >>>> export+import where you want to import 100.000 users. >>>> >>>> >>>> +1 >>>> >>>> >>>> As I mentioned in another mail few weeks ago, we can have: >>>> >>>> 1) Table with the progress (51.000 users already >>>> imported, around 49.000 remaining etc.) >>>> >>>> >>>> We would still need to split into multiple files in either >>>> case. Having a single json file with 100K users is probably >>>> not going to perform very well. So what I proposed would >>>> actually work for long-running import as well. If each file >>>> has a manageable amount of users (say ~5 min to import) >>>> then each file will be marked as imported or failed. At >>>> least for now I don't think we should do smaller batches >>>> than one file. As long as one file is imported within the >>>> same TX then it's an all or nothing import. >>>> >>>> 2) Concurrency and dividing the work among cluster >>>> nodes (Node1 will import 50.000 users and node2 another >>>> 50.000 users) >>>> >>>> >>>> This would be solved as well. Each node picks up a file >>>> that's not processed yet. Marks it in the DB and then gets >>>> to process it. >>>> >>>> 3) Failover (Import won't be completely broken if >>>> cluster node crashes after import 90.000, but can >>>> continue on other cluster nodes) >>>> >>>> I think the stuff I did recently for pre-loading >>>> offline sessions at startup could be reused for this >>>> stuff too and it can handle (2) and (3) . Also it can >>>> handle parallel import triggered from more cluster nodes. >>>> >>>> For example: currently if you trigger kubernetes with 2 >>>> cluster nodes, both nodes will start to import same >>>> file at the same time because import triggered by node1 >>>> is not yet finished before node2 is started, so there >>>> is not yet existing DB record that file is already >>>> imported. With the stuff I did, just the coordinator >>>> (node1) will start the import . Node2 will wait until >>>> import triggered by node1 is finished, but at the same >>>> time it can "help" to import some users (pages) if >>>> coordinator asks him to do so. This impl is based on >>>> infinispan distributed executor service >>>> http://infinispan.org/docs/5.3.x/user_guide/user_guide.html#_distributed_execution_framework >>>> . >>>> >>>> >>>> The DB record needs to be created before a node tries to >>>> import it, including a timestamp when it started the >>>> import. It should then be updated once the import is >>>> completed, with the result. Using the distributed execution >>>> framework sounds like a good idea though. How do you >>>> prevent scheduling the same job multiple times? For example >>>> if all nodes on startup scan the import folder and simply >>>> import everything they find, then there will be multiple of >>>> the same job. Not really a big deal as the first thing the >>>> job should do is check if there's a record in the DB already. >>> With distributed executor, it's the cluster coordinator, >>> which coordinates which node would import what. It will send >>> messages to cluster nodes like "Hey, please import the file >>> testrealm-users-3.json with timestamp abcd123" . >>> >>> After node finishes the job, it notifies coordinator and >>> coordinator will insert DB record and mark it as finished. >>> So there is no DB record inserted before node starts import, >>> because whole coordination is handled by the coordinator. >>> Also there will never be same file imported more times by >>> different cluster nodes. >>> >>> Only exception would be if cluster node crashes before >>> import is finished. Then it needs to be reimported by other >>> cluster node, but that's the case with DB locks as well. >>> >>> IMO the DB locks approach doesn't handle well crash of some >>> cluster node. For example when node2 crashes unexpectedly >>> when it's importing the file testrealm-users-3.json, the DB >>> lock is held by this node, so other cluster nodes can't >>> start on importing the file (until timeout occurs.) >>> >>> On the other hand, distributed executor approach may have >>> issues if there is inconsistent content of the >>> standalone/import directory among cluster nodes. However it >>> can be solved, so that each node will need to send checksums >>> of the files it has and coordinator will need to ensure that >>> file with checksum "abcd123" is assigned just to the node >>> which has this file. >>> >>> >>> With Docker/Kubernetes all nodes would have the same files. At >>> least initially. Would be nice if we could come up with a >>> solution where you can just drop an additional file onto any >>> node and have it imported. >> Exactly, was thinking about Docker too. Here we don't have any >> issue at all. >> >> The main question here is, do we want to support the scenario >> when various cluster nodes have different content? As I >> mentioned, distributed coordinator can handle it, so that each >> cluster node will send the checksums of the files it has and >> coordinator will always assign to node just the checksums, which >> it has. >> >> However regardless of distributed executor approach or DB locks >> approach, there may be still the issues. For example: >> 1) The file testrealm.json with checksum "abc" is triggered for >> import on node1 >> 2) At the same time, admin will do some minor change in this file >> on node2 and save it. This will mean that checksum of the file on >> node2 will be changed to "def" >> 3) Node2 will trigger import of that file. So we have both node1 >> and node2 importing same file concurrently because the previously >> retrieved lock was for "abc" checksum, but now checksum is "def" >> >> This problem will be with both DB lock and DistributedExecutor >> approaches though... > Possible solution for this issue is, that when import is already > in progress, the newly added or changed checksums will be ignored. > The checksums will be always checked just at start of the import. > > Marek > >> >> Marek >>> >>> >>> >>> Marek >>> >>>> >>>> >>>> Marek >>>> >>>> >>>>> >>>>> >>>>> On Mon, Nov 9, 2015 at 1:20 PM, Stian Thorgersen >>>>> > >>>>> wrote: >>>>> >>>>> Currently we support importing a complete >>>>> realm definition using the import/export >>>>> feature. Issues with the current approach is: >>>>> >>>>> * Only complete realm - not possible to add to >>>>> an existing realm >>>>> * No good feedback if import was successful or not >>>>> * Use of system properties to initiate the >>>>> import is not very user friendly >>>>> * Not very elegant for provisioning. For >>>>> example a Docker image that want's to bundle >>>>> some initial setup ends up always running the >>>>> import of a realm, which is skipped if realm >>>>> exists >>>>> >>>>> To solve this I've come up with the following >>>>> proposal: >>>>> >>>>> Allow dropping representations to be imported >>>>> into 'standalone/import'. This should support >>>>> creating a new realm as well as importing into >>>>> an existing realm. When importing into an >>>>> existing realm we will have an import strategy >>>>> that is used to configure what happens if a >>>>> resource exists (user, role, identity >>>>> provider, user federtation provider). The >>>>> import strategies are: >>>>> >>>>> * Skip - existing resources are skipped, >>>>> * Fail - if any resource exists nothing is >>>>> imported >>>>> * Overwrite - any existing resources are deleted. >>>>> >>>>> The directory will be scanned at startup, but >>>>> there will also be an option to monitor this >>>>> directory at runtime. >>>>> >>>>> To prevent a file being imported multiple >>>>> times (also to make sure only one node in a >>>>> cluster imports) we will have a table in the >>>>> database that contains what files was >>>>> imported, from what node, date and result >>>>> (including a list of what resources where >>>>> imported, which was not, and stack trace if >>>>> applicable). The primary key will be the >>>>> checksum of the file. We will also add marker >>>>> files (.imported or >>>> file>.failed). The contents of the marker >>>>> files will be a json object with date >>>>> imported, outcome (including stack trace if >>>>> applicable) as well as a complete list of what >>>>> resources was successfully imported, what >>>>> where not. >>>>> >>>>> The files will also allow resolving system >>>>> properties and environment variables. For example: >>>>> >>>>> { >>>>> "secret": "${env.MYCLIENT_SECRET}" >>>>> } >>>>> >>>>> This will be very convenient for example with >>>>> Docker as it would be very easy to create a >>>>> Docker image that extends ours to add a few >>>>> clients and users. >>>>> >>>>> It will also be convenient for examples as it >>>>> will make it possible to add the required >>>>> clients and users to an existing realm. >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> >>> >>> >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/021bffe2/attachment-0001.html From sthorger at redhat.com Thu Nov 12 06:37:13 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 12 Nov 2015 12:37:13 +0100 Subject: [keycloak-dev] Cluster security In-Reply-To: References: Message-ID: AUTH in JGroups only authenticates nodes that join a group. Nodes will discard messages that are not sent by a member of the group. However, someone could in theory eavesdrop on the messages and pick up the id assigned to a member of the group, they could then send a message masquerading as a member of the group. We don't send any sensitive information through the cluster so even if someone was able to read messages sent to the cluster they would not be able to obtain anything valuable. Further there's very little a malicious node could do by sending fake messages to the cluster. With regards to realms, clients and users they would only be able to send invalidation messages, forcing nodes to re-load the data from the database. For sessions they could in theory create new user sessions, but these would not have any use as cookies and tokens are signed by the realms private key, so the fake sessions wouldn't have any use. In summary enabling auth should be sufficient and encryption should not be required as long as you use the default clustering setup (invalidation-cache for realm, users and clients). On 12 November 2015 at 08:52, Stian Thorgersen wrote: > I believe enabling auth is sufficient, but I'm not 100% sure. I've sent an > email to JGroups mailing list to confirm and I'll let you know. > > On 11 November 2015 at 23:05, Matthew Casperson < > matthew.casperson at autogeneral.com.au> wrote: > >> The docs state that "By default there's nothing to prevent unauthorized >> nodes from joining the cluster and sending potentially malicious messages >> to the cluster." ( >> http://keycloak.github.io/docs/userguide/keycloak-server/html/clustering.html >> ) >> >> Is this still the case if the jgroups stack in Wildfly has implemented >> the AUTH protocol? For example, the Openshift Wildfly config looks >> something like this: >> >> >> >> ${env.OPENSHIFT_GEAR_DNS} >> ${env.OPENSHIFT_WILDFLY_CLUSTER_PROXY_PORT} >> ${env.OPENSHIFT_WILDFLY_CLUSTER_PORT} >> ${env.OPENSHIFT_WILDFLY_IP} >> true >> >> >> 30000 >> ${env.OPENSHIFT_WILDFLY_CLUSTER} >> 0 >> 1 >> >> >> >> >> >> >> >> >> >> org.jgroups.auth.MD5Token >> SHA >> ${env.OPENSHIFT_SECRET_TOKEN} >> >> >> >> >> >> >> >> >> >> >> -- >> *Matthew Casperson* >> *Senior Front End Developer* >> Technology, Space & Distribution >> Auto & General Holdings Pty Ltd >> P: 07) 3377 8751 (Direct: 3377 8751) >> F: 07) 3377 8833 >> >> >> >> This email is sent by Auto & General Insurance Company Ltd, Auto & General Services Pty Ltd, Auto & General Holdings Pty Ltd or a related body corporate (Auto & General) and is for the intended addressee. >> The views expressed in this email and attachments (email) reflect the views of the stated author but may not reflect views of Auto & General. This email is confidential and subject to copyright. >> It may be privileged. If you are not the intended addressee, confidentiality and privilege have not been waived and any use, interference with, or disclosure of this email is unauthorised. >> If you are not the intended addressee please immediately notify the sender and then delete the email. Auto & General does not warrant that this email is error or virus free. >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/20ee49ef/attachment.html From erik.mulder at docdatapayments.com Thu Nov 12 07:58:08 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Thu, 12 Nov 2015 13:58:08 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B076@DDPEX01.DDP.dcloud.local> On 11/11/15 13:54, Stian Thorgersen wrote: Would you be interested in contributing this feature? ATM we don't have anyone available that could work on it. A contribution would also need to include functional tests and documentation. Yes, I'd like to contribute this feature. I'm not sure about the timeline though. I hope to be able to do it as part of our current project, but I might have to use my spare time as well. Is there some kind of deadline to be included in a certain release version? If so I'm happy with going down the route of using the Hibernate specific classes. The remaining issue is figuring out how to deal with classloading. Looks like the following should work: * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider I've done this and it works fine, successfully providing the extra classes to the EntityManagerFactory build process in DefaultJpaConnectionProviderFactory. * JpaEntityProvider should have a single method "Class getEntities" Yes, only we need some kind of collection type, so you can provide multiple entity classes per provider. I guess you were intending this, considering the plural name 'getEntities'. I suggest either Collection> or Set> depending on what is most consistent with the rest of the system. Do you have a preference? * Implement org.hibernate.boot.registry.classloading.spi.ClassLoaderService - looks like this can just return null for everything except classForName where it would return the classes returned by the JpaEntityProvider implementations I see no way to interfere in the creation of the ClassLoaderService. The official way is using the BootstrapServiceRegistryBuilder, but with the JPA / EntityManagerFactoryBuilderImpl route this happens 'out of reach'. I did find another way that works just as well: you can provide a 'custom' classloader to the Bootstrap.getEntityManagerFactoryBuilder. We can define a classloader that will return the extra JpaEntityProvider classes if requested. Only tricky part here is that Hibernate not only calls loadClass on a classloader, but before that also getResource to get a URL with an InputStream to the class bytes. It uses that to scan for annotations with Jandex. I fixed this by forwarding that request to the ClassLoader of the JpaEntityProvider provided class (through Class.getClassLoader()). This works fine and shouldn't be problem for any drop in jars. I can imagine though that if you use some exotic ProviderLoader(Factory), you might somehow get in trouble if the class byte[] is not available anymore after class loading. But this is a problem with the way Hibernate works, not with the way we extend Hibernate in this case. So I think it's fine to have a warning about this in the documentation, since it will probably never be a real problem. If you consider this as a no-go, please let me know. Last question I have considers the Hibernate version of KeyCloak. Currently it's 4.3.10, are there any plans to upgrade to 5? The code related to classloading etc is refactored considerably in Hibernate 5. So it would be a shame to fully get it working for 4.3.10, only to have to upgrade soon after that. I didn't look into the details of Hibernate 5 and I think the solution we came up with should remain more or less intact, but you never know, so that's why I ask. On 7 November 2015 at 23:39, Erik Mulder > wrote: On 06/11/15 14:46, Stian Thorgersen wrote: > We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? I was a little quick to state that with Hibernate you can add extra entity class names besides the one in persistence.xml, since I spotted a few answers on StackOverflow that said it could be done. But they resolve around classpath scanning or using a Spring managed Hibernate. Then I thought: 'if Spring can do it, I can do it too' so I investigated the Hibernate source code 'behind' Persistence.createEntityManagerFactory(unitName, properties). After some digging it turns out it's pretty simple to get extra class names in the configuration. See code sample below. The only problem is that Hibernate will only find classes that are part of the 'main' KeyCloak application, because of the way the Wildfly module system and ClassLoader strategy work. The debugger showed me Hibernate has these 3 class loaders available to look for classes: 1. ModuleClassLoader for Module "deployment.keycloak-server.war:main" from Service Module Loader 2. ModuleClassLoader for Module "org.hibernate:main" from local module loader 3. sun.misc.Launcher$AppClassLoader Number 1 has all other KeyCloak modules in it, so the entity classes from model-jpa will be found, but the wildfly-extensions module is missing, so entities in classes in a jar in the providers folder cannot be found. Now you guys obviously know a lot more about these internals, but as currently configured, it seems to me there is no way to let Hibernate 'see' these extra classes, since only the KeyCloak services module has a dependency on wildfly-extensions. So I think it boils down to these decisions: A. Do you accept a non-pure-JPA way of building the EntityManagerFactory that has some ties to the Hibernate internals? B. If A is no, than we're done. If yes, then you must find some way to get the extra configured classes 'into' Hibernate. You could get the wildfly-extensions module into scope of the Hibernate classloading. There are serveral ways to configure Hibernate classloading or you could flip some switches / dependencies in the module configuration. Another alternative is to create a separate 'dropfolder' besides themes and providers for JPA extensions, like 'models' or so and have that one be on the Hibernate classpath. But I don't know the exact design principles behind KeyCloak or the Wildfly module system. So maybe you have a better solution or maybe you conclude that this is 'not done' in terms of the architecture. Either way, I'd really appreciate some feedback on this and some thoughts on whether this could be a possible addition to KeyCloak in your eyes. Thanks, Erik Current JPA way. No way to 'interfere': emf = Persistence.createEntityManagerFactory(unitName, properties); Alternative Hibernate only way with adding extra entity class names: // Let Hibernate find and parse all 'persistence.xml' files found on the classpath. List persistenceUnits = PersistenceXmlParser.locatePersistenceUnits(properties); // Assume there is only one persistence unit found and that is the one we need. This can be made more robust by checking on the persistence unit name. ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = persistenceUnits.get(0); // Add extra class names. These could come from a 'JPA class name provider' SPI or something alike. persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity", "org.keycloak.models.jpa.entities.MerchantEntity"); // Let Hibernate create an EntityManagerFactory out of the (enriched) persistence unit configuration. emf = Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, properties).build(); On 6 November 2015 at 14:29, Erik Mulder <erik.mulder at docdatapayments.com> wrote: Thanks for pointing me explicitly to the SPI documentation. Of course that is exactly what I was looking for in my original question. I don't know how I overlooked this earlier! Probably I was not picking it up, because of almost a decade of developing on Spring projects, where this type of thing works differently. :-) I tried a quick test with a jar with one extra ProtocolMapper configured, put it in the providers folder and it worked like a charm! As for the JPA: We'll probably go with your suggestion of the separate EntityManagerFactory. Indeed there seems to be no way to 'programmatically extend' the list of entity classes in JPA besides editing or overwriting the persistence.xml. As you probably know it can be done in Hibernate, but I guess KeyCloak wants to stick to a generic JPA solution. That said, we might consider the Hibernate specific solution for our case, since being able to switch the JPA provider is not a requirement for us. And keeping the same connection/transaction is a lot easier in reasoning and debugging. We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? On 05/11/15 10:52, Stian Thorgersen wrote: The way to extend Keycloak is by implementing your own custom providers of the many SPIs we provide. Some SPIs are more stable (so marked as public) and others are not (so marked as private). If there are things that you want to customize that can't be done with an existing SPI then let us know and we may consider adding additional SPIs. On 4 November 2015 at 17:16, Erik Mulder <erik.mulder at docdatapayments.com> wrote: Thanks for your response! Indeed we already did a proof of concept where we added a custom mapper the way you described (didn't know it was 'protected' territory :). The question is: do we have to override the file 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper in the original project or is there another way where we don't need to touch the original sources and keep all our changes in a separate project? And how can we do it such that it stays easy to upgrade to newer KeyCloak releases? Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at the docs (http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) and examples for other provider (https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory) As for JPA: it would be easier to integrate with the existing JPA project. Again we are wondering whether to start modifying original sources (like persistence.xml) or try to 'externalize' our changes somehow and integrate them using existing 'hooks' in the system or maybe merge projects during build. Maybe there is no good answer to this and we'll always be having some manual merge pains when upgrading to new KeyCloak versions. We just wanted to check if there are preferred ways to add functionality with the least amount of impact on the original sources. I initially wanted the ability to add custom entities to the JpaConnectProvider, but couldn't find a way to define entities programatically with JPA. To add your own persistence.xml you would have to define your own implementation of JpaConnectionProvider and change what is loaded by default (connectionsJpa/provider attribute in keycloak-server.json). Alternative, which is cleaner, but you end up with separate connection/transaction, is to create your own EntityManagerFactory. If it's only used by one provider (for example a custom UserFederationProvider) there's no need to add a connect provider (that's just a way to share one EntityManagerFactory between multiple providers) and you can just create it in the MyUserFederationProviderFactory. On 04/11/15 15:30, Bill Burke wrote: > Custom mappers should be possible. I didn't document it as I wasn't > sure if we wanted to make the SPI public. Custom mappers should just > follow the Provider SPI and they will be picked up. If you see the > META-INF/services/... file in the resources directory of the "services" > or "broker" modules you'll see how to set this up. > > As for extending the JPA datamodel, what you could do is write a new JPA > Connections Provider and plug that in. See connections/jpa. I'm not > sure how you would handle the liquibase db migration. > > On 11/4/2015 6:03 AM, Erik Mulder wrote: >> Hi everybody, >> >> Quick intro: I?m part of a development team in The Netherlands that is >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >> this and will use OpenID Connect to secure our REST services. It?s a >> great product and seems to be the only one having both support for all >> kinds of security standards and a model and GUI for users and roles. >> Thanks for creating it! J >> >> (if this should be asked instead on the users mailing list, please >> correct me and I?ll post it there) >> >> So far, so good, but we have some extra requirements that do not fit >> into the base KeyCloak data model. See below for details if you?re >> interested. My question is: what is the preferred way / best practice to >> extend the functionality of KeyCloak while keeping the impact on the >> original sources to a minimum? Of course we could just fork the most >> recent version and start hacking away, but we?d like to be able to >> upgrade to newer versions of KeyCloak without too much hassle. >> Possibilities that we?ve come up with so far: >> >> 1.Create completely separate modules that will extend the functionality >> the way we need. >> >> 2.Fork on Github, apply custom changes, and try to merge in updates from >> the master / release branches / tags >> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >> as Truezip >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >> manipulate zip files by adding/removing/replacing or Shade >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple >> jars to 1 'uber-jar' containing the contents of both and when >> overlapping decide on conflicts through configuration. >> >> Of course number 1 is preferred, but I do not see how to add custom >> mappers or JPA entities without making changes in the original module >> files. The other options seem like valid alternatives, but maybe there >> is better / standard way to do this. So any help / insight / shared >> experience on this is much appreciated, thanks! >> >> Kind regards, >> >> Erik Mulder >> >> Senior Software Engineer >> >> Docdata Payments ? NL >> >> P.S. Details on why we want to extend the KeyCloak data model: (any >> feedback on the contents of this P.S. is also welcome!) >> >> Our clients are merchants that have several webshops. We manage their >> online payments (shopping cart checkout). We want to be able to let a >> merchant manage their own users and let a user have different roles for >> different webshops within the same merchant. The overall possible roles >> are fixed though, no specific roles per merchant. We could create a >> separate realm for every merchant, but then we need to duplicate all >> roles every time. Furthermore, in KeyCloak there is no concept of a role >> within a certain context. This is very understandable, since every >> situation has it?s own requirements. We did a proof of concept by adding >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >> and adding a custom mapper that can put this information on the Access >> token. Worked like a charm! But it does need some changes in the >> KeyCloak modules and sources to work, hence the question above. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/6d154dec/attachment-0001.html From sthorger at redhat.com Thu Nov 12 08:32:01 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 12 Nov 2015 14:32:01 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml In-Reply-To: <9A5619B792BBA041AE094585791BB71C0137B668B076@DDPEX01.DDP.dcloud.local> References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B076@DDPEX01.DDP.dcloud.local> Message-ID: On 12 November 2015 at 13:58, Erik Mulder wrote: > On 11/11/15 13:54, Stian Thorgersen wrote: > > Would you be interested in contributing this feature? ATM we don't have > anyone available that could work on it. A contribution would also need to > include functional tests and documentation. > > > Yes, I'd like to contribute this feature. I'm not sure about the timeline > though. I hope to be able to do it as part of our current project, but I > might have to use my spare time as well. Is there some kind of deadline to > be included in a certain release version? > We do a release every ~6 weeks. It's already a bit late for 1.7 (it's due end of the month) so would have to aim for 1.8 in either case (early January). > > > If so I'm happy with going down the route of using the Hibernate specific > classes. The remaining issue is figuring out how to deal with classloading. > > Looks like the following should work: > > * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider > > > I've done this and it works fine, successfully providing the extra classes > to the EntityManagerFactory build process in > DefaultJpaConnectionProviderFactory. > > * JpaEntityProvider should have a single method "Class getEntities" > > > Yes, only we need some kind of collection type, so you can provide > multiple entity classes per provider. I guess you were intending this, > considering the plural name 'getEntities'. I suggest either > Collection> or Set> depending on what is most > consistent with the rest of the system. Do you have a preference? > Yup - List would be fine, that's what we tend to use as it's nicer to use than collection or set. > > > * Implement org.hibernate.boot.registry.classloading.spi.ClassLoaderService > - looks like this can just return null for everything except classForName > where it would return the classes returned by the JpaEntityProvider > implementations > > > I see no way to interfere in the creation of the ClassLoaderService. The > official way is using the BootstrapServiceRegistryBuilder, but with the > JPA / EntityManagerFactoryBuilderImpl route this happens 'out of reach'. > I did find another way that works just as well: you can provide a 'custom' > classloader to the Bootstrap.getEntityManagerFactoryBuilder. We can > define a classloader that will return the extra JpaEntityProvider classes > if requested. Only tricky part here is that Hibernate not only calls > loadClass on a classloader, but before that also getResource to get a URL > with an InputStream to the class bytes. It uses that to scan for > annotations with Jandex. I fixed this by forwarding that request to the > ClassLoader of the JpaEntityProvider provided class (through > Class.getClassLoader()). This works fine and shouldn't be problem for any > drop in jars. I can imagine though that if you use some exotic > ProviderLoader(Factory), you might somehow get in trouble if the class > byte[] is not available anymore after class loading. But this is a problem > with the way Hibernate works, not with the way we extend Hibernate in this > case. So I think it's fine to have a warning about this in the > documentation, since it will probably never be a real problem. If you > consider this as a no-go, please let me know. > Sounds OK, but not sure what you mean about exotic ProviderLoader(Factory) is that a Hibernate thing? > > > Last question I have considers the Hibernate version of KeyCloak. > Currently it's 4.3.10, are there any plans to upgrade to 5? The code > related to classloading etc is refactored considerably in Hibernate 5. So > it would be a shame to fully get it working for 4.3.10, only to have to > upgrade soon after that. I didn't look into the details of Hibernate 5 and > I think the solution we came up with should remain more or less intact, but > you never know, so that's why I ask. > We are soon moving to WildFly 10 which includes Hibernate 5, but we still need to support EAP 6.4 which includes Hibernate 4. At some point next year we will drop support for EAP 6.4 and move on to EAP 7. We either have to support both Hibernate 4 and 5 for a while, or we make it use the old approach on Hibernate 4 (so now custom entity class support on EAP 6.4) and the new approach on Hibernate 5. That would probably require some magic reflection code though. > > > > On 7 November 2015 at 23:39, Erik Mulder > wrote: > >> *On 06/11/15 14:46, Stian Thorgersen wrote:* >> *> We could use Hibernate directly to boostrap as long as it can return >> an EntityManager. Do you know if that's possible?* >> >> I was a little quick to state that with Hibernate you can add extra >> entity class names besides the one in persistence.xml, since I spotted a >> few answers on StackOverflow that said it could be done. But they resolve >> around classpath scanning or using a Spring managed Hibernate. Then I >> thought: 'if Spring can do it, I can do it too' so I investigated the >> Hibernate source code 'behind' >> Persistence.createEntityManagerFactory(unitName, properties). After some >> digging it turns out it's pretty simple to get extra class names in the >> configuration. See code sample below. >> >> The only problem is that Hibernate will only find classes that are part >> of the 'main' KeyCloak application, because of the way the Wildfly module >> system and ClassLoader strategy work. The debugger showed me Hibernate has >> these 3 class loaders available to look for classes: >> 1. ModuleClassLoader for Module "deployment.keycloak-server.war:main" >> from Service Module Loader >> 2. ModuleClassLoader for Module "org.hibernate:main" from local module >> loader >> 3. sun.misc.Launcher$AppClassLoader >> >> Number 1 has all other KeyCloak modules in it, so the entity classes from >> model-jpa will be found, but the wildfly-extensions module is missing, so >> entities in classes in a jar in the providers folder cannot be found. Now >> you guys obviously know a lot more about these internals, but as currently >> configured, it seems to me there is no way to let Hibernate 'see' these >> extra classes, since only the KeyCloak services module has a dependency on >> wildfly-extensions. >> >> So I think it boils down to these decisions: >> A. Do you accept a non-pure-JPA way of building the EntityManagerFactory >> that has some ties to the Hibernate internals? >> B. If A is no, than we're done. If yes, then you must find some way to >> get the extra configured classes 'into' Hibernate. You could get the >> wildfly-extensions module into scope of the Hibernate classloading. There >> are serveral ways to configure Hibernate classloading or you could flip >> some switches / dependencies in the module configuration. Another >> alternative is to create a separate 'dropfolder' besides themes and >> providers for JPA extensions, like 'models' or so and have that one be on >> the Hibernate classpath. But I don't know the exact design principles >> behind KeyCloak or the Wildfly module system. So maybe you have a better >> solution or maybe you conclude that this is 'not done' in terms of the >> architecture. >> >> Either way, I'd really appreciate some feedback on this and some thoughts >> on whether this could be a possible addition to KeyCloak in your eyes. >> >> Thanks, Erik >> >> >> Current JPA way. No way to 'interfere': >> emf = Persistence.createEntityManagerFactory(unitName, properties); >> >> Alternative Hibernate only way with adding extra entity class names: >> // Let Hibernate find and parse all 'persistence.xml' files found on the >> classpath. >> List persistenceUnits = >> PersistenceXmlParser.locatePersistenceUnits(properties); >> // Assume there is only one persistence unit found and that is the one we >> need. This can be made more robust by checking on the persistence unit name. >> ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = >> persistenceUnits.get(0); >> // Add extra class names. These could come from a 'JPA class name >> provider' SPI or something alike. >> >> persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity", >> "org.keycloak.models.jpa.entities.MerchantEntity"); >> // Let Hibernate create an EntityManagerFactory out of the (enriched) >> persistence unit configuration. >> emf = Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, >> properties).build(); >> >> >> >> >> On 6 November 2015 at 14:29, Erik Mulder < >> erik.mulder at docdatapayments.com> wrote: >> >>> Thanks for pointing me explicitly to the SPI documentation. Of course >>> that is exactly what I was looking for in my original question. I don't >>> know how I overlooked this earlier! Probably I was not picking it up, >>> because of almost a decade of developing on Spring projects, where this >>> type of thing works differently. :-) >>> >>> I tried a quick test with a jar with one extra ProtocolMapper >>> configured, put it in the providers folder and it worked like a charm! >>> >>> As for the JPA: We'll probably go with your suggestion of the separate >>> EntityManagerFactory. Indeed there seems to be no way to 'programmatically >>> extend' the list of entity classes in JPA besides editing or overwriting >>> the persistence.xml. As you probably know it can be done in Hibernate, but >>> I guess KeyCloak wants to stick to a generic JPA solution. That said, we >>> might consider the Hibernate specific solution for our case, since being >>> able to switch the JPA provider is not a requirement for us. And keeping >>> the same connection/transaction is a lot easier in reasoning and debugging. >>> >> >> We could use Hibernate directly to boostrap as long as it can return an >> EntityManager. Do you know if that's possible? >> >> >>> >>> >>> >>> On 05/11/15 10:52, Stian Thorgersen wrote: >>> >>> The way to extend Keycloak is by implementing your own custom providers >>> of the many SPIs we provide. Some SPIs are more stable (so marked as >>> public) and others are not (so marked as private). If there are things that >>> you want to customize that can't be done with an existing SPI then let us >>> know and we may consider adding additional SPIs. >>> >>> On 4 November 2015 at 17:16, Erik Mulder < >>> erik.mulder at docdatapayments.com> wrote: >>> >>>> Thanks for your response! >>>> >>>> Indeed we already did a proof of concept where we added a custom mapper >>>> the way you described (didn't know it was 'protected' territory :). The >>>> question is: do we have to override the file >>>> 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper >>>> in the original project or is there another way where we don't need to >>>> touch the original sources and keep all our changes in a separate >>>> project? And how can we do it such that it stays easy to upgrade to >>>> newer KeyCloak releases? >>>> >>> >>> Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look >>> at the docs ( >>> >>> http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) >>> and examples for other provider ( >>> >>> https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory >>> ) >>> >>> >>>> >>>> As for JPA: it would be easier to integrate with the existing JPA >>>> project. Again we are wondering whether to start modifying original >>>> sources (like persistence.xml) or try to 'externalize' our changes >>>> somehow and integrate them using existing 'hooks' in the system or maybe >>>> merge projects during build. >>>> >>>> Maybe there is no good answer to this and we'll always be having some >>>> manual merge pains when upgrading to new KeyCloak versions. We just >>>> wanted to check if there are preferred ways to add functionality with >>>> the least amount of impact on the original sources. >>>> >>> >>> I initially wanted the ability to add custom entities to the >>> JpaConnectProvider, but couldn't find a way to define entities >>> programatically with JPA. To add your own persistence.xml you would have to >>> define your own implementation of JpaConnectionProvider and change what is >>> loaded by default (connectionsJpa/provider attribute in >>> keycloak-server.json). >>> >>> Alternative, which is cleaner, but you end up with separate >>> connection/transaction, is to create your own EntityManagerFactory. If it's >>> only used by one provider (for example a custom UserFederationProvider) >>> there's no need to add a connect provider (that's just a way to share one >>> EntityManagerFactory between multiple providers) and you can just create it >>> in the MyUserFederationProviderFactory. >>> >>> >>>> >>>> >>>> On 04/11/15 15:30, Bill Burke wrote: >>>> > Custom mappers should be possible. I didn't document it as I wasn't >>>> > sure if we wanted to make the SPI public. Custom mappers should just >>>> > follow the Provider SPI and they will be picked up. If you see the >>>> > META-INF/services/... file in the resources directory of the >>>> "services" >>>> > or "broker" modules you'll see how to set this up. >>>> > >>>> > As for extending the JPA datamodel, what you could do is write a new >>>> JPA >>>> > Connections Provider and plug that in. See connections/jpa. I'm not >>>> > sure how you would handle the liquibase db migration. >>>> > >>>> > On 11/4/2015 6:03 AM, Erik Mulder wrote: >>>> >> Hi everybody, >>>> >> >>>> >> Quick intro: I?m part of a development team in The Netherlands that >>>> is >>>> >> building a company-wide SSO solution. We?ve chosen KeyCloak to >>>> realize >>>> >> this and will use OpenID Connect to secure our REST services. It?s a >>>> >> great product and seems to be the only one having both support for >>>> all >>>> >> kinds of security standards and a model and GUI for users and roles. >>>> >> Thanks for creating it! J >>>> >> >>>> >> (if this should be asked instead on the users mailing list, please >>>> >> correct me and I?ll post it there) >>>> >> >>>> >> So far, so good, but we have some extra requirements that do not fit >>>> >> into the base KeyCloak data model. See below for details if you?re >>>> >> interested. My question is: what is the preferred way / best >>>> practice to >>>> >> extend the functionality of KeyCloak while keeping the impact on the >>>> >> original sources to a minimum? Of course we could just fork the most >>>> >> recent version and start hacking away, but we?d like to be able to >>>> >> upgrade to newer versions of KeyCloak without too much hassle. >>>> >> Possibilities that we?ve come up with so far: >>>> >> >>>> >> 1.Create completely separate modules that will extend the >>>> functionality >>>> >> the way we need. >>>> >> >>>> >> 2.Fork on Github, apply custom changes, and try to merge in updates >>>> from >>>> >> the master / release branches / tags >>>> >> >>>> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, >>>> such >>>> >> as Truezip >>>> >> ( >>>> http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >>>> >> manipulate zip files by adding/removing/replacing or Shade >>>> >> ( >>>> http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple >>>> >> jars to 1 'uber-jar' containing the contents of both and when >>>> >> overlapping decide on conflicts through configuration. >>>> >> >>>> >> Of course number 1 is preferred, but I do not see how to add custom >>>> >> mappers or JPA entities without making changes in the original module >>>> >> files. The other options seem like valid alternatives, but maybe >>>> there >>>> >> is better / standard way to do this. So any help / insight / shared >>>> >> experience on this is much appreciated, thanks! >>>> >> >>>> >> Kind regards, >>>> >> >>>> >> Erik Mulder >>>> >> >>>> >> Senior Software Engineer >>>> >> >>>> >> Docdata Payments ? NL >>>> >> >>>> >> P.S. Details on why we want to extend the KeyCloak data model: (any >>>> >> feedback on the contents of this P.S. is also welcome!) >>>> >> >>>> >> Our clients are merchants that have several webshops. We manage their >>>> >> online payments (shopping cart checkout). We want to be able to let a >>>> >> merchant manage their own users and let a user have different roles >>>> for >>>> >> different webshops within the same merchant. The overall possible >>>> roles >>>> >> are fixed though, no specific roles per merchant. We could create a >>>> >> separate realm for every merchant, but then we need to duplicate all >>>> >> roles every time. Furthermore, in KeyCloak there is no concept of a >>>> role >>>> >> within a certain context. This is very understandable, since every >>>> >> situation has it?s own requirements. We did a proof of concept by >>>> adding >>>> >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >>>> >> and adding a custom mapper that can put this information on the >>>> Access >>>> >> token. Worked like a charm! But it does need some changes in the >>>> >> KeyCloak modules and sources to work, hence the question above. >>>> >> >>>> >> >>>> >> >>>> >> _______________________________________________ >>>> >> keycloak-dev mailing list >>>> >> keycloak-dev at lists.jboss.org >>>> >> >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>> >>> >>> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/8472016b/attachment-0001.html From christian.beikov at gmail.com Thu Nov 12 08:59:10 2015 From: christian.beikov at gmail.com (Christian Beikov) Date: Thu, 12 Nov 2015 14:59:10 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml In-Reply-To: References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B076@DDPEX01.DDP.dcloud.local> Message-ID: <56449B2E.1060604@gmail.com> I just wanted to mention that you actually don't have to create the EntityManagerFactory yourselves to do that. Hibernate Envers adds entities to the persistence unit too. Maybe ask someone from the hibernate team for details, but I think your entry point would be the Integrator: https://github.com/hibernate/hibernate-orm/blob/2964ecbf1b74003fcc337b0574487c724638fe94/hibernate-envers/src/main/java/org/hibernate/envers/boot/internal/EnversIntegrator.java Mit freundlichen Gr??en, ------------------------------------------------------------------------ *Christian Beikov* Am 12.11.2015 um 14:32 schrieb Stian Thorgersen: > > > On 12 November 2015 at 13:58, Erik Mulder > > wrote: > > On 11/11/15 13:54, Stian Thorgersen wrote: >> Would you be interested in contributing this feature? ATM we >> don't have anyone available that could work on it. A contribution >> would also need to include functional tests and documentation. > > Yes, I'd like to contribute this feature. I'm not sure about the > timeline though. I hope to be able to do it as part of our current > project, but I might have to use my spare time as well. Is there > some kind of deadline to be included in a certain release version? > > > We do a release every ~6 weeks. It's already a bit late for 1.7 (it's > due end of the month) so would have to aim for 1.8 in either case > (early January). > > > >> If so I'm happy with going down the route of using the Hibernate >> specific classes. The remaining issue is figuring out how to deal >> with classloading. >> >> Looks like the following should work: >> >> * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider > > I've done this and it works fine, successfully providing the extra > classes to the EntityManagerFactory build process in > DefaultJpaConnectionProviderFactory. > >> * JpaEntityProvider should have a single method "Class >> getEntities" > > Yes, only we need some kind of collection type, so you can provide > multiple entity classes per provider. I guess you were intending > this, considering the plural name 'getEntities'. I suggest either > Collection> or Set> depending on what is most > consistent with the rest of the system. Do you have a preference? > > > Yup - List would be fine, that's what we tend to use as it's nicer to > use than collection or set. > > > >> * Implement >> org.hibernate.boot.registry.classloading.spi.ClassLoaderService - >> looks like this can just return null for everything >> except classForName where it would return the classes returned by >> the JpaEntityProvider implementations > > I see no way to interfere in the creation of the > ClassLoaderService. The official way is using the > BootstrapServiceRegistryBuilder, but with the JPA / > EntityManagerFactoryBuilderImpl route this happens 'out of reach'. > I did find another way that works just as well: you can provide a > 'custom' classloader to the > Bootstrap.getEntityManagerFactoryBuilder. We can define a > classloader that will return the extra JpaEntityProvider classes > if requested. Only tricky part here is that Hibernate not only > calls loadClass on a classloader, but before that also getResource > to get a URL with an InputStream to the class bytes. It uses that > to scan for annotations with Jandex. I fixed this by forwarding > that request to the ClassLoader of the JpaEntityProvider provided > class (through Class.getClassLoader()). This works fine and > shouldn't be problem for any drop in jars. I can imagine though > that if you use some exotic ProviderLoader(Factory), you might > somehow get in trouble if the class byte[] is not available > anymore after class loading. But this is a problem with the way > Hibernate works, not with the way we extend Hibernate in this > case. So I think it's fine to have a warning about this in the > documentation, since it will probably never be a real problem. If > you consider this as a no-go, please let me know. > > > Sounds OK, but not sure what you mean about > exotic ProviderLoader(Factory) is that a Hibernate thing? > > > > Last question I have considers the Hibernate version of KeyCloak. > Currently it's 4.3.10, are there any plans to upgrade to 5? The > code related to classloading etc is refactored considerably in > Hibernate 5. So it would be a shame to fully get it working for > 4.3.10, only to have to upgrade soon after that. I didn't look > into the details of Hibernate 5 and I think the solution we came > up with should remain more or less intact, but you never know, so > that's why I ask. > > > We are soon moving to WildFly 10 which includes Hibernate 5, but we > still need to support EAP 6.4 which includes Hibernate 4. At some > point next year we will drop support for EAP 6.4 and move on to EAP 7. > > We either have to support both Hibernate 4 and 5 for a while, or we > make it use the old approach on Hibernate 4 (so now custom entity > class support on EAP 6.4) and the new approach on Hibernate 5. That > would probably require some magic reflection code though. > > > > >> On 7 November 2015 at 23:39, Erik Mulder >> > > wrote: >> >> /On 06/11/15 14:46, Stian Thorgersen wrote:// >> //> We could use Hibernate directly to boostrap as long as it >> can return an EntityManager. Do you know if that's possible?// >> / >> I was a little quick to state that with Hibernate you can add >> extra entity class names besides the one in persistence.xml, >> since I spotted a few answers on StackOverflow that said it >> could be done. But they resolve around classpath scanning or >> using a Spring managed Hibernate. Then I thought: 'if Spring >> can do it, I can do it too' so I investigated the Hibernate >> source code >> 'behind'Persistence.createEntityManagerFactory(unitName, >> properties). After some digging it turns out it's pretty >> simple to get extra class names in the configuration. See >> code sample below. >> >> The only problem is that Hibernate will only find classes >> that are part of the 'main' KeyCloak application, because of >> the way the Wildfly module system and ClassLoader strategy >> work. The debugger showed me Hibernate has these 3 class >> loaders available to look for classes: >> 1. ModuleClassLoader for Module >> "deployment.keycloak-server.war:main" from Service Module Loader >> 2. ModuleClassLoader for Module "org.hibernate:main" from >> local module loader >> 3. sun.misc.Launcher$AppClassLoader >> >> Number 1 has all other KeyCloak modules in it, so the entity >> classes from model-jpa will be found, but the >> wildfly-extensions module is missing, so entities in classes >> in a jar in the providers folder cannot be found. Now you >> guys obviously know a lot more about these internals, but as >> currently configured, it seems to me there is no way to let >> Hibernate 'see' these extra classes, since only the KeyCloak >> services module has a dependency on wildfly-extensions. >> >> So I think it boils down to these decisions: >> A. Do you accept a non-pure-JPA way of building the >> EntityManagerFactory that has some ties to the Hibernate >> internals? >> B. If A is no, than we're done. If yes, then you must find >> some way to get the extra configured classes 'into' >> Hibernate. You could get the wildfly-extensions module into >> scope of the Hibernate classloading. There are serveral ways >> to configure Hibernate classloading or you could flip some >> switches / dependencies in the module configuration. Another >> alternative is to create a separate 'dropfolder' besides >> themes and providers for JPA extensions, like 'models' or so >> and have that one be on the Hibernate classpath. But I don't >> know the exact design principles behind KeyCloak or the >> Wildfly module system. So maybe you have a better solution or >> maybe you conclude that this is 'not done' in terms of the >> architecture. >> >> Either way, I'd really appreciate some feedback on this and >> some thoughts on whether this could be a possible addition to >> KeyCloak in your eyes. >> >> Thanks, Erik >> >> >> Current JPA way. No way to 'interfere': >> emf = Persistence.createEntityManagerFactory(unitName, >> properties); >> >> Alternative Hibernate only way with adding extra entity class >> names: >> // Let Hibernate find and parse all 'persistence.xml' files >> found on the classpath. >> List persistenceUnits = >> PersistenceXmlParser.locatePersistenceUnits(properties); >> // Assume there is only one persistence unit found and that >> is the one we need. This can be made more robust by checking >> on the persistence unit name. >> ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = >> persistenceUnits.get(0); >> // Add extra class names. These could come from a 'JPA class >> name provider' SPI or something alike. >> persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity","org.keycloak.models.jpa.entities.MerchantEntity"); >> // Let Hibernate create an EntityManagerFactory out of the >> (enriched) persistence unit configuration. >> emf = >> Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, >> properties).build(); >> >> >> >> >> On 6 November 2015 at 14:29, Erik Mulder >> > > wrote: >> >> Thanks for pointing me explicitly to the SPI >> documentation. Of course that is exactly what I was >> looking for in my original question. I don't know how I >> overlooked this earlier! Probably I was not picking it >> up, because of almost a decade of developing on Spring >> projects, where this type of thing works differently. :-) >> >> I tried a quick test with a jar with one extra >> ProtocolMapper configured, put it in the providers folder >> and it worked like a charm! >> >> As for the JPA: We'll probably go with your suggestion of >> the separate EntityManagerFactory. Indeed there seems to >> be no way to 'programmatically extend' the list of entity >> classes in JPA besides editing or overwriting the >> persistence.xml. As you probably know it can be done in >> Hibernate, but I guess KeyCloak wants to stick to a >> generic JPA solution. That said, we might consider the >> Hibernate specific solution for our case, since being >> able to switch the JPA provider is not a requirement for >> us. And keeping the same connection/transaction is a lot >> easier in reasoning and debugging. >> >> >> We could use Hibernate directly to boostrap as long as it can >> return an EntityManager. Do you know if that's possible? >> >> >> >> >> On 05/11/15 10:52, Stian Thorgersen wrote: >>> The way to extend Keycloak is by implementing your own >>> custom providers of the many SPIs we provide. Some SPIs >>> are more stable (so marked as public) and others are not >>> (so marked as private). If there are things that you >>> want to customize that can't be done with an existing >>> SPI then let us know and we may consider adding >>> additional SPIs. >>> >>> On 4 November 2015 at 17:16, Erik Mulder >>> >> > wrote: >>> >>> Thanks for your response! >>> >>> Indeed we already did a proof of concept where we >>> added a custom mapper >>> the way you described (didn't know it was >>> 'protected' territory :). The >>> question is: do we have to override the file >>> 'org.keycloak.protocol.ProtocolMapper' for this and >>> add the new mapper >>> in the original project or is there another way >>> where we don't need to >>> touch the original sources and keep all our changes >>> in a separate >>> project? And how can we do it such that it stays >>> easy to upgrade to >>> newer KeyCloak releases? >>> >>> >>> Each jar has it's own >>> org.keycloak.protocol.ProtocolMapper. Take a look at the >>> docs >>> (http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) >>> and examples for other provider >>> (https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory) >>> >>> >>> As for JPA: it would be easier to integrate with the >>> existing JPA >>> project. Again we are wondering whether to start >>> modifying original >>> sources (like persistence.xml) or try to >>> 'externalize' our changes >>> somehow and integrate them using existing 'hooks' in >>> the system or maybe >>> merge projects during build. >>> >>> Maybe there is no good answer to this and we'll >>> always be having some >>> manual merge pains when upgrading to new KeyCloak >>> versions. We just >>> wanted to check if there are preferred ways to add >>> functionality with >>> the least amount of impact on the original sources. >>> >>> >>> I initially wanted the ability to add custom entities to >>> the JpaConnectProvider, but couldn't find a way to >>> define entities programatically with JPA. To add your >>> own persistence.xml you would have to define your own >>> implementation of JpaConnectionProvider and change what >>> is loaded by default (connectionsJpa/provider attribute >>> in keycloak-server.json). >>> >>> Alternative, which is cleaner, but you end up with >>> separate connection/transaction, is to create your own >>> EntityManagerFactory. If it's only used by one provider >>> (for example a custom UserFederationProvider) there's no >>> need to add a connect provider (that's just a way to >>> share one EntityManagerFactory between multiple >>> providers) and you can just create it in the >>> MyUserFederationProviderFactory. >>> >>> >>> >>> On 04/11/15 15:30, Bill Burke wrote: >>> > Custom mappers should be possible. I didn't >>> document it as I wasn't >>> > sure if we wanted to make the SPI public. Custom >>> mappers should just >>> > follow the Provider SPI and they will be picked >>> up. If you see the >>> > META-INF/services/... file in the resources >>> directory of the "services" >>> > or "broker" modules you'll see how to set this up. >>> > >>> > As for extending the JPA datamodel, what you could >>> do is write a new JPA >>> > Connections Provider and plug that in. See >>> connections/jpa. I'm not >>> > sure how you would handle the liquibase db migration. >>> > >>> > On 11/4/2015 6:03 AM, Erik Mulder wrote: >>> >> Hi everybody, >>> >> >>> >> Quick intro: I?m part of a development team in >>> The Netherlands that is >>> >> building a company-wide SSO solution. We?ve >>> chosen KeyCloak to realize >>> >> this and will use OpenID Connect to secure our >>> REST services. It?s a >>> >> great product and seems to be the only one having >>> both support for all >>> >> kinds of security standards and a model and GUI >>> for users and roles. >>> >> Thanks for creating it! J >>> >> >>> >> (if this should be asked instead on the users >>> mailing list, please >>> >> correct me and I?ll post it there) >>> >> >>> >> So far, so good, but we have some extra >>> requirements that do not fit >>> >> into the base KeyCloak data model. See below for >>> details if you?re >>> >> interested. My question is: what is the preferred >>> way / best practice to >>> >> extend the functionality of KeyCloak while >>> keeping the impact on the >>> >> original sources to a minimum? Of course we could >>> just fork the most >>> >> recent version and start hacking away, but we?d >>> like to be able to >>> >> upgrade to newer versions of KeyCloak without too >>> much hassle. >>> >> Possibilities that we?ve come up with so far: >>> >> >>> >> 1.Create completely separate modules that will >>> extend the functionality >>> >> the way we need. >>> >> >>> >> 2.Fork on Github, apply custom changes, and try >>> to merge in updates from >>> >> the master / release branches / tags >>> >> >>> >> 3.Apply custom changes on KeyCloak artifacts >>> using a Maven plugin, such >>> >> as Truezip >>> >> >>> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) >>> - >>> >> manipulate zip files by adding/removing/replacing >>> or Shade >>> >> >>> (http://maven.apache.org/plugins/maven-shade-plugin/) - >>> combine multiple >>> >> jars to 1 'uber-jar' containing the contents of >>> both and when >>> >> overlapping decide on conflicts through >>> configuration. >>> >> >>> >> Of course number 1 is preferred, but I do not see >>> how to add custom >>> >> mappers or JPA entities without making changes in >>> the original module >>> >> files. The other options seem like valid >>> alternatives, but maybe there >>> >> is better / standard way to do this. So any help >>> / insight / shared >>> >> experience on this is much appreciated, thanks! >>> >> >>> >> Kind regards, >>> >> >>> >> Erik Mulder >>> >> >>> >> Senior Software Engineer >>> >> >>> >> Docdata Payments ? NL >>> >> >>> >> P.S. Details on why we want to extend the >>> KeyCloak data model: (any >>> >> feedback on the contents of this P.S. is also >>> welcome!) >>> >> >>> >> Our clients are merchants that have several >>> webshops. We manage their >>> >> online payments (shopping cart checkout). We want >>> to be able to let a >>> >> merchant manage their own users and let a user >>> have different roles for >>> >> different webshops within the same merchant. The >>> overall possible roles >>> >> are fixed though, no specific roles per merchant. >>> We could create a >>> >> separate realm for every merchant, but then we >>> need to duplicate all >>> >> roles every time. Furthermore, in KeyCloak there >>> is no concept of a role >>> >> within a certain context. This is very >>> understandable, since every >>> >> situation has it?s own requirements. We did a >>> proof of concept by adding >>> >> tables and entities for Merchant, UserMerchant, >>> UserMerchantRole etc. >>> >> and adding a custom mapper that can put this >>> information on the Access >>> >> token. Worked like a charm! But it does need some >>> changes in the >>> >> KeyCloak modules and sources to work, hence the >>> question above. >>> >> >>> >> >>> >> >>> >> _______________________________________________ >>> >> keycloak-dev mailing list >>> >> keycloak-dev at lists.jboss.org >>> >>> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >> >> >> >> > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/0cafa0f4/attachment-0001.html From sthorger at redhat.com Thu Nov 12 10:07:23 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 12 Nov 2015 16:07:23 +0100 Subject: [keycloak-dev] Client registration Message-ID: I've prepared some details about what we are planning around dynamic client registration in Keycloak. This will hopefully be included in 1.7. Slides are here: https://docs.google.com/presentation/d/1kQe3pjjACiQUDoCW5RGPzJwBHn1OIQ2mxn1K_NnDGOc/edit?usp=sharing -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/c142fd6e/attachment.html From erik.mulder at docdatapayments.com Thu Nov 12 11:14:09 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Thu, 12 Nov 2015 17:14:09 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B076@DDPEX01.DDP.dcloud.local> Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B07A@DDPEX01.DDP.dcloud.local> @Christian Beikov: Thanks for the hint on integrators! I was afraid I could not use integrators when working with the JPA 'route', but I found a way. There is a property EntityManagerFactoryBuilderImpl.INTEGRATOR_PROVIDER that you can use to supply custom integrators. This even works with the original 'pure' JPA call: Persistence.createEntityManagerFactory. In the integrator you can add annotated classes to the Hibernate Configuration. So the solution is still Hibernate only of course. The only downside is that there is less validation on the supplied classes, they are just added to the config directly. For instance I tested with adding Object.class as annotated class. This doesn't raise an exception and seems to be silently ignored by Hibernate. Not sure if it might result in problems during runtime though. Either way, normally this should be fine and with a little extra documentation on how these extra classes are handled I think using an integrator is the least intrusive thus best way to go. Agreed? Ok, List as collection type is fine too, indeed easier to use. Conceptually I like Set, because List seems to imply there is some kind of ordering involved which in this case there isn't. But that's just a minor matter of taste. By exotic ProviderLoader(Factory) I meant the org.keycloak.provider.ProviderLoader(Factory) that you can use to load Provider(Factory)'s from other locations than a file system based classpath. If that were to be a 'read once' kindof location (like some stream) there could be a problem. But with the integrator solution that shouldn't matter anymore, since it's just the Class object that we need. As for Hibernate 4/5, I'll try to make a solution that works for both Hibernate 4 and 5 in the same way. If that's not possible, I guess we could have separate code paths for 4 and 5, depending on the runtime version. I hope there is an easy way of figuring this out, maybe some static Hibernate class holding a version or so. A quick Google returns some useful results, so I'm sure we'll get that sorted if needed. End of the month is probably too soon indeed, so let's aim for 1.8. Should I (or someone else) create one (or several) JIRA tickets for this? On 12/11/15 14:32, Stian Thorgersen wrote: On 12 November 2015 at 13:58, Erik Mulder > wrote: On 11/11/15 13:54, Stian Thorgersen wrote: Would you be interested in contributing this feature? ATM we don't have anyone available that could work on it. A contribution would also need to include functional tests and documentation. Yes, I'd like to contribute this feature. I'm not sure about the timeline though. I hope to be able to do it as part of our current project, but I might have to use my spare time as well. Is there some kind of deadline to be included in a certain release version? We do a release every ~6 weeks. It's already a bit late for 1.7 (it's due end of the month) so would have to aim for 1.8 in either case (early January). If so I'm happy with going down the route of using the Hibernate specific classes. The remaining issue is figuring out how to deal with classloading. Looks like the following should work: * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider I've done this and it works fine, successfully providing the extra classes to the EntityManagerFactory build process in DefaultJpaConnectionProviderFactory. * JpaEntityProvider should have a single method "Class getEntities" Yes, only we need some kind of collection type, so you can provide multiple entity classes per provider. I guess you were intending this, considering the plural name 'getEntities'. I suggest either Collection> or Set> depending on what is most consistent with the rest of the system. Do you have a preference? Yup - List would be fine, that's what we tend to use as it's nicer to use than collection or set. * Implement org.hibernate.boot.registry.classloading.spi.ClassLoaderService - looks like this can just return null for everything except classForName where it would return the classes returned by the JpaEntityProvider implementations I see no way to interfere in the creation of the ClassLoaderService. The official way is using the BootstrapServiceRegistryBuilder, but with the JPA / EntityManagerFactoryBuilderImpl route this happens 'out of reach'. I did find another way that works just as well: you can provide a 'custom' classloader to the Bootstrap.getEntityManagerFactoryBuilder. We can define a classloader that will return the extra JpaEntityProvider classes if requested. Only tricky part here is that Hibernate not only calls loadClass on a classloader, but before that also getResource to get a URL with an InputStream to the class bytes. It uses that to scan for annotations with Jandex. I fixed this by forwarding that request to the ClassLoader of the JpaEntityProvider provided class (through Class.getClassLoader()). This works fine and shouldn't be problem for any drop in jars. I can imagine though that if you use some exotic ProviderLoader(Factory), you might somehow get in trouble if the class byte[] is not available anymore after class loading. But this is a problem with the way Hibernate works, not with the way we extend Hibernate in this case. So I think it's fine to have a warning about this in the documentation, since it will probably never be a real problem. If you consider this as a no-go, please let me know. Sounds OK, but not sure what you mean about exotic ProviderLoader(Factory) is that a Hibernate thing? Last question I have considers the Hibernate version of KeyCloak. Currently it's 4.3.10, are there any plans to upgrade to 5? The code related to classloading etc is refactored considerably in Hibernate 5. So it would be a shame to fully get it working for 4.3.10, only to have to upgrade soon after that. I didn't look into the details of Hibernate 5 and I think the solution we came up with should remain more or less intact, but you never know, so that's why I ask. We are soon moving to WildFly 10 which includes Hibernate 5, but we still need to support EAP 6.4 which includes Hibernate 4. At some point next year we will drop support for EAP 6.4 and move on to EAP 7. We either have to support both Hibernate 4 and 5 for a while, or we make it use the old approach on Hibernate 4 (so now custom entity class support on EAP 6.4) and the new approach on Hibernate 5. That would probably require some magic reflection code though. On 7 November 2015 at 23:39, Erik Mulder <erik.mulder at docdatapayments.com> wrote: On 06/11/15 14:46, Stian Thorgersen wrote: > We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? I was a little quick to state that with Hibernate you can add extra entity class names besides the one in persistence.xml, since I spotted a few answers on StackOverflow that said it could be done. But they resolve around classpath scanning or using a Spring managed Hibernate. Then I thought: 'if Spring can do it, I can do it too' so I investigated the Hibernate source code 'behind' Persistence.createEntityManagerFactory(unitName, properties). After some digging it turns out it's pretty simple to get extra class names in the configuration. See code sample below. The only problem is that Hibernate will only find classes that are part of the 'main' KeyCloak application, because of the way the Wildfly module system and ClassLoader strategy work. The debugger showed me Hibernate has these 3 class loaders available to look for classes: 1. ModuleClassLoader for Module "deployment.keycloak-server.war:main" from Service Module Loader 2. ModuleClassLoader for Module "org.hibernate:main" from local module loader 3. sun.misc.Launcher$AppClassLoader Number 1 has all other KeyCloak modules in it, so the entity classes from model-jpa will be found, but the wildfly-extensions module is missing, so entities in classes in a jar in the providers folder cannot be found. Now you guys obviously know a lot more about these internals, but as currently configured, it seems to me there is no way to let Hibernate 'see' these extra classes, since only the KeyCloak services module has a dependency on wildfly-extensions. So I think it boils down to these decisions: A. Do you accept a non-pure-JPA way of building the EntityManagerFactory that has some ties to the Hibernate internals? B. If A is no, than we're done. If yes, then you must find some way to get the extra configured classes 'into' Hibernate. You could get the wildfly-extensions module into scope of the Hibernate classloading. There are serveral ways to configure Hibernate classloading or you could flip some switches / dependencies in the module configuration. Another alternative is to create a separate 'dropfolder' besides themes and providers for JPA extensions, like 'models' or so and have that one be on the Hibernate classpath. But I don't know the exact design principles behind KeyCloak or the Wildfly module system. So maybe you have a better solution or maybe you conclude that this is 'not done' in terms of the architecture. Either way, I'd really appreciate some feedback on this and some thoughts on whether this could be a possible addition to KeyCloak in your eyes. Thanks, Erik Current JPA way. No way to 'interfere': emf = Persistence.createEntityManagerFactory(unitName, properties); Alternative Hibernate only way with adding extra entity class names: // Let Hibernate find and parse all 'persistence.xml' files found on the classpath. List persistenceUnits = PersistenceXmlParser.locatePersistenceUnits(properties); // Assume there is only one persistence unit found and that is the one we need. This can be made more robust by checking on the persistence unit name. ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = persistenceUnits.get(0); // Add extra class names. These could come from a 'JPA class name provider' SPI or something alike. persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity", "org.keycloak.models.jpa.entities.MerchantEntity"); // Let Hibernate create an EntityManagerFactory out of the (enriched) persistence unit configuration. emf = Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, properties).build(); On 6 November 2015 at 14:29, Erik Mulder <erik.mulder at docdatapayments.com> wrote: Thanks for pointing me explicitly to the SPI documentation. Of course that is exactly what I was looking for in my original question. I don't know how I overlooked this earlier! Probably I was not picking it up, because of almost a decade of developing on Spring projects, where this type of thing works differently. :-) I tried a quick test with a jar with one extra ProtocolMapper configured, put it in the providers folder and it worked like a charm! As for the JPA: We'll probably go with your suggestion of the separate EntityManagerFactory. Indeed there seems to be no way to 'programmatically extend' the list of entity classes in JPA besides editing or overwriting the persistence.xml. As you probably know it can be done in Hibernate, but I guess KeyCloak wants to stick to a generic JPA solution. That said, we might consider the Hibernate specific solution for our case, since being able to switch the JPA provider is not a requirement for us. And keeping the same connection/transaction is a lot easier in reasoning and debugging. We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? On 05/11/15 10:52, Stian Thorgersen wrote: The way to extend Keycloak is by implementing your own custom providers of the many SPIs we provide. Some SPIs are more stable (so marked as public) and others are not (so marked as private). If there are things that you want to customize that can't be done with an existing SPI then let us know and we may consider adding additional SPIs. On 4 November 2015 at 17:16, Erik Mulder <erik.mulder at docdatapayments.com> wrote: Thanks for your response! Indeed we already did a proof of concept where we added a custom mapper the way you described (didn't know it was 'protected' territory :). The question is: do we have to override the file 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper in the original project or is there another way where we don't need to touch the original sources and keep all our changes in a separate project? And how can we do it such that it stays easy to upgrade to newer KeyCloak releases? Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at the docs (http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) and examples for other provider (https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory) As for JPA: it would be easier to integrate with the existing JPA project. Again we are wondering whether to start modifying original sources (like persistence.xml) or try to 'externalize' our changes somehow and integrate them using existing 'hooks' in the system or maybe merge projects during build. Maybe there is no good answer to this and we'll always be having some manual merge pains when upgrading to new KeyCloak versions. We just wanted to check if there are preferred ways to add functionality with the least amount of impact on the original sources. I initially wanted the ability to add custom entities to the JpaConnectProvider, but couldn't find a way to define entities programatically with JPA. To add your own persistence.xml you would have to define your own implementation of JpaConnectionProvider and change what is loaded by default (connectionsJpa/provider attribute in keycloak-server.json). Alternative, which is cleaner, but you end up with separate connection/transaction, is to create your own EntityManagerFactory. If it's only used by one provider (for example a custom UserFederationProvider) there's no need to add a connect provider (that's just a way to share one EntityManagerFactory between multiple providers) and you can just create it in the MyUserFederationProviderFactory. On 04/11/15 15:30, Bill Burke wrote: > Custom mappers should be possible. I didn't document it as I wasn't > sure if we wanted to make the SPI public. Custom mappers should just > follow the Provider SPI and they will be picked up. If you see the > META-INF/services/... file in the resources directory of the "services" > or "broker" modules you'll see how to set this up. > > As for extending the JPA datamodel, what you could do is write a new JPA > Connections Provider and plug that in. See connections/jpa. I'm not > sure how you would handle the liquibase db migration. > > On 11/4/2015 6:03 AM, Erik Mulder wrote: >> Hi everybody, >> >> Quick intro: I?m part of a development team in The Netherlands that is >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >> this and will use OpenID Connect to secure our REST services. It?s a >> great product and seems to be the only one having both support for all >> kinds of security standards and a model and GUI for users and roles. >> Thanks for creating it! J >> >> (if this should be asked instead on the users mailing list, please >> correct me and I?ll post it there) >> >> So far, so good, but we have some extra requirements that do not fit >> into the base KeyCloak data model. See below for details if you?re >> interested. My question is: what is the preferred way / best practice to >> extend the functionality of KeyCloak while keeping the impact on the >> original sources to a minimum? Of course we could just fork the most >> recent version and start hacking away, but we?d like to be able to >> upgrade to newer versions of KeyCloak without too much hassle. >> Possibilities that we?ve come up with so far: >> >> 1.Create completely separate modules that will extend the functionality >> the way we need. >> >> 2.Fork on Github, apply custom changes, and try to merge in updates from >> the master / release branches / tags >> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >> as Truezip >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >> manipulate zip files by adding/removing/replacing or Shade >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple >> jars to 1 'uber-jar' containing the contents of both and when >> overlapping decide on conflicts through configuration. >> >> Of course number 1 is preferred, but I do not see how to add custom >> mappers or JPA entities without making changes in the original module >> files. The other options seem like valid alternatives, but maybe there >> is better / standard way to do this. So any help / insight / shared >> experience on this is much appreciated, thanks! >> >> Kind regards, >> >> Erik Mulder >> >> Senior Software Engineer >> >> Docdata Payments ? NL >> >> P.S. Details on why we want to extend the KeyCloak data model: (any >> feedback on the contents of this P.S. is also welcome!) >> >> Our clients are merchants that have several webshops. We manage their >> online payments (shopping cart checkout). We want to be able to let a >> merchant manage their own users and let a user have different roles for >> different webshops within the same merchant. The overall possible roles >> are fixed though, no specific roles per merchant. We could create a >> separate realm for every merchant, but then we need to duplicate all >> roles every time. Furthermore, in KeyCloak there is no concept of a role >> within a certain context. This is very understandable, since every >> situation has it?s own requirements. We did a proof of concept by adding >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >> and adding a custom mapper that can put this information on the Access >> token. Worked like a charm! But it does need some changes in the >> KeyCloak modules and sources to work, hence the question above. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/10e205ac/attachment-0001.html From christian.beikov at gmail.com Thu Nov 12 11:19:50 2015 From: christian.beikov at gmail.com (Christian Beikov) Date: Thu, 12 Nov 2015 17:19:50 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml In-Reply-To: <9A5619B792BBA041AE094585791BB71C0137B668B07A@DDPEX01.DDP.dcloud.local> References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B076@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B07A@DDPEX01.DDP.dcloud.local> Message-ID: <5644BC26.60602@gmail.com> You need different code for Hibernate 4 and 5 because the integrator interfaces are not compatible. The integrator is a service that is loaded through a java.util.ServiceLoader so I am not sure why you would need the property. Mit freundlichen Gr??en, ------------------------------------------------------------------------ *Christian Beikov* Am 12.11.2015 um 17:14 schrieb Erik Mulder: > /@Christian Beikov/: Thanks for the hint on integrators! > > I was afraid I could not use integrators when working with the JPA > 'route', but I found a way. There is a property > EntityManagerFactoryBuilderImpl.INTEGRATOR_PROVIDER that you can use > to supply custom integrators. This even works with the original 'pure' > JPA call: Persistence.createEntityManagerFactory. In the integrator > you can add annotated classes to the Hibernate Configuration. So the > solution is still Hibernate only of course. > > The only downside is that there is less validation on the supplied > classes, they are just added to the config directly. For instance I > tested with adding Object.class as annotated class. This doesn't raise > an exception and seems to be silently ignored by Hibernate. Not sure > if it might result in problems during runtime though. Either way, > normally this should be fine and with a little extra documentation on > how these extra classes are handled I think using an integrator is the > least intrusive thus best way to go. Agreed? > > Ok, List as collection type is fine too, indeed easier to use. > Conceptually I like Set, because List seems to imply there is some > kind of ordering involved which in this case there isn't. But that's > just a minor matter of taste. > > By exotic ProviderLoader(Factory) I meant the > org.keycloak.provider.ProviderLoader(Factory) that you can use to load > Provider(Factory)'s from other locations than a file system based > classpath. If that were to be a 'read once' kindof location (like some > stream) there could be a problem. But with the integrator solution > that shouldn't matter anymore, since it's just the Class object that > we need. > > As for Hibernate 4/5, I'll try to make a solution that works for both > Hibernate 4 and 5 in the same way. If that's not possible, I guess we > could have separate code paths for 4 and 5, depending on the runtime > version. I hope there is an easy way of figuring this out, maybe some > static Hibernate class holding a version or so. A quick Google returns > some useful results, so I'm sure we'll get that sorted if needed. > > End of the month is probably too soon indeed, so let's aim for 1.8. > Should I (or someone else) create one (or several) JIRA tickets for this? > > > On 12/11/15 14:32, Stian Thorgersen wrote: >> >> >> On 12 November 2015 at 13:58, Erik Mulder >> > > wrote: >> >> On 11/11/15 13:54, Stian Thorgersen wrote: >>> Would you be interested in contributing this feature? ATM we >>> don't have anyone available that could work on it. A >>> contribution would also need to include functional tests and >>> documentation. >> >> Yes, I'd like to contribute this feature. I'm not sure about the >> timeline though. I hope to be able to do it as part of our >> current project, but I might have to use my spare time as well. >> Is there some kind of deadline to be included in a certain >> release version? >> >> >> We do a release every ~6 weeks. It's already a bit late for 1.7 (it's >> due end of the month) so would have to aim for 1.8 in either case >> (early January). >> >> >> >>> If so I'm happy with going down the route of using the Hibernate >>> specific classes. The remaining issue is figuring out how to >>> deal with classloading. >>> >>> Looks like the following should work: >>> >>> * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider >> >> I've done this and it works fine, successfully providing the >> extra classes to the EntityManagerFactory build process in >> DefaultJpaConnectionProviderFactory. >> >>> * JpaEntityProvider should have a single method "Class >>> getEntities" >> >> Yes, only we need some kind of collection type, so you can >> provide multiple entity classes per provider. I guess you were >> intending this, considering the plural name 'getEntities'. I >> suggest either Collection> or Set> depending on >> what is most consistent with the rest of the system. Do you have >> a preference? >> >> >> Yup - List would be fine, that's what we tend to use as it's nicer to >> use than collection or set. >> >> >> >>> * Implement >>> org.hibernate.boot.registry.classloading.spi.ClassLoaderService >>> - looks like this can just return null for everything >>> except classForName where it would return the classes returned >>> by the JpaEntityProvider implementations >> >> I see no way to interfere in the creation of the >> ClassLoaderService. The official way is using the >> BootstrapServiceRegistryBuilder, but with the JPA / >> EntityManagerFactoryBuilderImpl route this happens 'out of >> reach'. I did find another way that works just as well: you can >> provide a 'custom' classloader to the >> Bootstrap.getEntityManagerFactoryBuilder. We can define a >> classloader that will return the extra JpaEntityProvider classes >> if requested. Only tricky part here is that Hibernate not only >> calls loadClass on a classloader, but before that also >> getResource to get a URL with an InputStream to the class bytes. >> It uses that to scan for annotations with Jandex. I fixed this by >> forwarding that request to the ClassLoader of the >> JpaEntityProvider provided class (through >> Class.getClassLoader()). This works fine and shouldn't be problem >> for any drop in jars. I can imagine though that if you use some >> exotic ProviderLoader(Factory), you might somehow get in trouble >> if the class byte[] is not available anymore after class loading. >> But this is a problem with the way Hibernate works, not with the >> way we extend Hibernate in this case. So I think it's fine to >> have a warning about this in the documentation, since it will >> probably never be a real problem. If you consider this as a >> no-go, please let me know. >> >> >> Sounds OK, but not sure what you mean about >> exotic ProviderLoader(Factory) is that a Hibernate thing? >> >> >> >> Last question I have considers the Hibernate version of KeyCloak. >> Currently it's 4.3.10, are there any plans to upgrade to 5? The >> code related to classloading etc is refactored considerably in >> Hibernate 5. So it would be a shame to fully get it working for >> 4.3.10, only to have to upgrade soon after that. I didn't look >> into the details of Hibernate 5 and I think the solution we came >> up with should remain more or less intact, but you never know, so >> that's why I ask. >> >> >> We are soon moving to WildFly 10 which includes Hibernate 5, but we >> still need to support EAP 6.4 which includes Hibernate 4. At some >> point next year we will drop support for EAP 6.4 and move on to EAP 7. >> >> We either have to support both Hibernate 4 and 5 for a while, or we >> make it use the old approach on Hibernate 4 (so now custom entity >> class support on EAP 6.4) and the new approach on Hibernate 5. That >> would probably require some magic reflection code though. >> >> >> >> >>> On 7 November 2015 at 23:39, Erik Mulder >>> wrote: >>> >>> /On 06/11/15 14:46, Stian Thorgersen wrote:// >>> //> We could use Hibernate directly to boostrap as long as >>> it can return an EntityManager. Do you know if that's >>> possible?// >>> / >>> I was a little quick to state that with Hibernate you can >>> add extra entity class names besides the one in >>> persistence.xml, since I spotted a few answers on >>> StackOverflow that said it could be done. But they resolve >>> around classpath scanning or using a Spring managed >>> Hibernate. Then I thought: 'if Spring can do it, I can do it >>> too' so I investigated the Hibernate source code >>> 'behind'Persistence.createEntityManagerFactory(unitName, >>> properties). After some digging it turns out it's pretty >>> simple to get extra class names in the configuration. See >>> code sample below. >>> >>> The only problem is that Hibernate will only find classes >>> that are part of the 'main' KeyCloak application, because of >>> the way the Wildfly module system and ClassLoader strategy >>> work. The debugger showed me Hibernate has these 3 class >>> loaders available to look for classes: >>> 1. ModuleClassLoader for Module >>> "deployment.keycloak-server.war:main" from Service Module Loader >>> 2. ModuleClassLoader for Module "org.hibernate:main" from >>> local module loader >>> 3. sun.misc.Launcher$AppClassLoader >>> >>> Number 1 has all other KeyCloak modules in it, so the entity >>> classes from model-jpa will be found, but the >>> wildfly-extensions module is missing, so entities in classes >>> in a jar in the providers folder cannot be found. Now you >>> guys obviously know a lot more about these internals, but as >>> currently configured, it seems to me there is no way to let >>> Hibernate 'see' these extra classes, since only the KeyCloak >>> services module has a dependency on wildfly-extensions. >>> >>> So I think it boils down to these decisions: >>> A. Do you accept a non-pure-JPA way of building the >>> EntityManagerFactory that has some ties to the Hibernate >>> internals? >>> B. If A is no, than we're done. If yes, then you must find >>> some way to get the extra configured classes 'into' >>> Hibernate. You could get the wildfly-extensions module into >>> scope of the Hibernate classloading. There are serveral ways >>> to configure Hibernate classloading or you could flip some >>> switches / dependencies in the module configuration. Another >>> alternative is to create a separate 'dropfolder' besides >>> themes and providers for JPA extensions, like 'models' or so >>> and have that one be on the Hibernate classpath. But I don't >>> know the exact design principles behind KeyCloak or the >>> Wildfly module system. So maybe you have a better solution >>> or maybe you conclude that this is 'not done' in terms of >>> the architecture. >>> >>> Either way, I'd really appreciate some feedback on this and >>> some thoughts on whether this could be a possible addition >>> to KeyCloak in your eyes. >>> >>> Thanks, Erik >>> >>> >>> Current JPA way. No way to 'interfere': >>> emf = Persistence.createEntityManagerFactory(unitName, >>> properties); >>> >>> Alternative Hibernate only way with adding extra entity >>> class names: >>> // Let Hibernate find and parse all 'persistence.xml' files >>> found on the classpath. >>> List persistenceUnits = >>> PersistenceXmlParser.locatePersistenceUnits(properties); >>> // Assume there is only one persistence unit found and that >>> is the one we need. This can be made more robust by checking >>> on the persistence unit name. >>> ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = >>> persistenceUnits.get(0); >>> // Add extra class names. These could come from a 'JPA class >>> name provider' SPI or something alike. >>> persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity","org.keycloak.models.jpa.entities.MerchantEntity"); >>> // Let Hibernate create an EntityManagerFactory out of the >>> (enriched) persistence unit configuration. >>> emf = >>> Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, >>> properties).build(); >>> >>> >>> >>> >>> On 6 November 2015 at 14:29, Erik Mulder >>> wrote: >>> >>> Thanks for pointing me explicitly to the SPI >>> documentation. Of course that is exactly what I was >>> looking for in my original question. I don't know how I >>> overlooked this earlier! Probably I was not picking it >>> up, because of almost a decade of developing on Spring >>> projects, where this type of thing works differently. :-) >>> >>> I tried a quick test with a jar with one extra >>> ProtocolMapper configured, put it in the providers >>> folder and it worked like a charm! >>> >>> As for the JPA: We'll probably go with your suggestion >>> of the separate EntityManagerFactory. Indeed there seems >>> to be no way to 'programmatically extend' the list of >>> entity classes in JPA besides editing or overwriting the >>> persistence.xml. As you probably know it can be done in >>> Hibernate, but I guess KeyCloak wants to stick to a >>> generic JPA solution. That said, we might consider the >>> Hibernate specific solution for our case, since being >>> able to switch the JPA provider is not a requirement for >>> us. And keeping the same connection/transaction is a lot >>> easier in reasoning and debugging. >>> >>> >>> We could use Hibernate directly to boostrap as long as it >>> can return an EntityManager. Do you know if that's possible? >>> >>> >>> >>> >>> On 05/11/15 10:52, Stian Thorgersen wrote: >>>> The way to extend Keycloak is by implementing your own >>>> custom providers of the many SPIs we provide. Some SPIs >>>> are more stable (so marked as public) and others are >>>> not (so marked as private). If there are things that >>>> you want to customize that can't be done with an >>>> existing SPI then let us know and we may consider >>>> adding additional SPIs. >>>> >>>> On 4 November 2015 at 17:16, Erik Mulder >>>> wrote: >>>> >>>> Thanks for your response! >>>> >>>> Indeed we already did a proof of concept where we >>>> added a custom mapper >>>> the way you described (didn't know it was >>>> 'protected' territory :). The >>>> question is: do we have to override the file >>>> 'org.keycloak.protocol.ProtocolMapper' for this and >>>> add the new mapper >>>> in the original project or is there another way >>>> where we don't need to >>>> touch the original sources and keep all our changes >>>> in a separate >>>> project? And how can we do it such that it stays >>>> easy to upgrade to >>>> newer KeyCloak releases? >>>> >>>> >>>> Each jar has it's own >>>> org.keycloak.protocol.ProtocolMapper. Take a look at >>>> the docs >>>> (http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) >>>> and examples for other provider >>>> (https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory) >>>> >>>> >>>> As for JPA: it would be easier to integrate with >>>> the existing JPA >>>> project. Again we are wondering whether to start >>>> modifying original >>>> sources (like persistence.xml) or try to >>>> 'externalize' our changes >>>> somehow and integrate them using existing 'hooks' >>>> in the system or maybe >>>> merge projects during build. >>>> >>>> Maybe there is no good answer to this and we'll >>>> always be having some >>>> manual merge pains when upgrading to new KeyCloak >>>> versions. We just >>>> wanted to check if there are preferred ways to add >>>> functionality with >>>> the least amount of impact on the original sources. >>>> >>>> >>>> I initially wanted the ability to add custom entities >>>> to the JpaConnectProvider, but couldn't find a way to >>>> define entities programatically with JPA. To add your >>>> own persistence.xml you would have to define your own >>>> implementation of JpaConnectionProvider and change what >>>> is loaded by default (connectionsJpa/provider attribute >>>> in keycloak-server.json). >>>> >>>> Alternative, which is cleaner, but you end up with >>>> separate connection/transaction, is to create your own >>>> EntityManagerFactory. If it's only used by one provider >>>> (for example a custom UserFederationProvider) there's >>>> no need to add a connect provider (that's just a way to >>>> share one EntityManagerFactory between multiple >>>> providers) and you can just create it in the >>>> MyUserFederationProviderFactory. >>>> >>>> >>>> >>>> On 04/11/15 15:30, Bill Burke wrote: >>>> > Custom mappers should be possible. I didn't >>>> document it as I wasn't >>>> > sure if we wanted to make the SPI public. Custom >>>> mappers should just >>>> > follow the Provider SPI and they will be picked >>>> up. If you see the >>>> > META-INF/services/... file in the resources >>>> directory of the "services" >>>> > or "broker" modules you'll see how to set this up. >>>> > >>>> > As for extending the JPA datamodel, what you >>>> could do is write a new JPA >>>> > Connections Provider and plug that in. See >>>> connections/jpa. I'm not >>>> > sure how you would handle the liquibase db migration. >>>> > >>>> > On 11/4/2015 6:03 AM, Erik Mulder wrote: >>>> >> Hi everybody, >>>> >> >>>> >> Quick intro: I?m part of a development team in >>>> The Netherlands that is >>>> >> building a company-wide SSO solution. We?ve >>>> chosen KeyCloak to realize >>>> >> this and will use OpenID Connect to secure our >>>> REST services. It?s a >>>> >> great product and seems to be the only one >>>> having both support for all >>>> >> kinds of security standards and a model and GUI >>>> for users and roles. >>>> >> Thanks for creating it! J >>>> >> >>>> >> (if this should be asked instead on the users >>>> mailing list, please >>>> >> correct me and I?ll post it there) >>>> >> >>>> >> So far, so good, but we have some extra >>>> requirements that do not fit >>>> >> into the base KeyCloak data model. See below for >>>> details if you?re >>>> >> interested. My question is: what is the >>>> preferred way / best practice to >>>> >> extend the functionality of KeyCloak while >>>> keeping the impact on the >>>> >> original sources to a minimum? Of course we >>>> could just fork the most >>>> >> recent version and start hacking away, but we?d >>>> like to be able to >>>> >> upgrade to newer versions of KeyCloak without >>>> too much hassle. >>>> >> Possibilities that we?ve come up with so far: >>>> >> >>>> >> 1.Create completely separate modules that will >>>> extend the functionality >>>> >> the way we need. >>>> >> >>>> >> 2.Fork on Github, apply custom changes, and try >>>> to merge in updates from >>>> >> the master / release branches / tags >>>> >> >>>> >> 3.Apply custom changes on KeyCloak artifacts >>>> using a Maven plugin, such >>>> >> as Truezip >>>> >> >>>> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) >>>> - >>>> >> manipulate zip files by >>>> adding/removing/replacing or Shade >>>> >> >>>> (http://maven.apache.org/plugins/maven-shade-plugin/) >>>> - combine multiple >>>> >> jars to 1 'uber-jar' containing the contents of >>>> both and when >>>> >> overlapping decide on conflicts through >>>> configuration. >>>> >> >>>> >> Of course number 1 is preferred, but I do not >>>> see how to add custom >>>> >> mappers or JPA entities without making changes >>>> in the original module >>>> >> files. The other options seem like valid >>>> alternatives, but maybe there >>>> >> is better / standard way to do this. So any help >>>> / insight / shared >>>> >> experience on this is much appreciated, thanks! >>>> >> >>>> >> Kind regards, >>>> >> >>>> >> Erik Mulder >>>> >> >>>> >> Senior Software Engineer >>>> >> >>>> >> Docdata Payments ? NL >>>> >> >>>> >> P.S. Details on why we want to extend the >>>> KeyCloak data model: (any >>>> >> feedback on the contents of this P.S. is also >>>> welcome!) >>>> >> >>>> >> Our clients are merchants that have several >>>> webshops. We manage their >>>> >> online payments (shopping cart checkout). We >>>> want to be able to let a >>>> >> merchant manage their own users and let a user >>>> have different roles for >>>> >> different webshops within the same merchant. The >>>> overall possible roles >>>> >> are fixed though, no specific roles per >>>> merchant. We could create a >>>> >> separate realm for every merchant, but then we >>>> need to duplicate all >>>> >> roles every time. Furthermore, in KeyCloak there >>>> is no concept of a role >>>> >> within a certain context. This is very >>>> understandable, since every >>>> >> situation has it?s own requirements. We did a >>>> proof of concept by adding >>>> >> tables and entities for Merchant, UserMerchant, >>>> UserMerchantRole etc. >>>> >> and adding a custom mapper that can put this >>>> information on the Access >>>> >> token. Worked like a charm! But it does need >>>> some changes in the >>>> >> KeyCloak modules and sources to work, hence the >>>> question above. >>>> >> >>>> >> >>>> >> >>>> >> _______________________________________________ >>>> >> keycloak-dev mailing list >>>> >> keycloak-dev at lists.jboss.org >>>> >> >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> >>> >>> >>> >>> >> >> > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151112/599656af/attachment-0001.html From bburke at redhat.com Fri Nov 13 09:02:55 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 13 Nov 2015 09:02:55 -0500 Subject: [keycloak-dev] Why there is extra redirects now In-Reply-To: <56210C17.8020907@redhat.com> References: <56210C17.8020907@redhat.com> Message-ID: <5645ED8F.8090605@redhat.com> -------- Forwarded Message -------- Subject: [keycloak-dev] changes to browser-based flows Date: Fri, 16 Oct 2015 10:39:19 -0400 From: Bill Burke To: keycloak-dev at lists.jboss.org FYI: Not important if you are uninterested in design. Prior to my last commit if you hit the browser refresh button you would either have had the authentication flow completely reset or received an error page. Also, changing the local on some required actions pages would end up in an error condition. So...To fix this I made some changes to browser based flows: * After any successful action processing (i.e. a form POST), the browser is sent a 302 redirect to a "safer" page. If you are in the authentication phase, then this redirect will be to /authenticate?code={code}, registration /register?code={code}, reset credentials /reset-credentials?code={code}, required actions /required-action?code={code}. When these URIs are executed, Keycloak will figure out where the user is in the flow and render things appropriately. * After authentication, the browser will be 302 redirected to /required-action?code={code} The reason for these changes is to support when the user clicks the browser refresh button. The refresh button will resubmit the previous request. Prior to this change there were issues with this. For example, previously, if there was a required action and you just logged in via username and password, the URI in the browser would still point to the username/password page even though the required action page was being rendered. If the refresh button was hit, the previous username password POST would be resent to the username/password page, Keycloak would say "WTF are you doing?!?" and abort. There were similar issues like this everywhere. Other things effected by this fix: * required actions no longer change the ACTION_KEY or the ClientSessionModel.getAction(). * ClientSessionModel.getAction() will either be AUTHENTICATION, REQUIRED_ACTIONS, EXECUTE_ACTIONS, LOGGED_OUT, or OAUTH_GRANT. * After authentication, the flow manager will change the action from AUTHENTICATION to REQUIRED_ACTIONS. Overall, this is less performant as there are additional HTTP redirect requests being thrown in, but should provide a better user experience. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Fri Nov 13 09:07:39 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 13 Nov 2015 09:07:39 -0500 Subject: [keycloak-dev] Why there is extra redirects now In-Reply-To: <5645ED8F.8090605@redhat.com> References: <56210C17.8020907@redhat.com> <5645ED8F.8090605@redhat.com> Message-ID: <5645EEAB.4040501@redhat.com> I'll take another stab at this. On 11/13/2015 9:02 AM, Bill Burke wrote: > > > > -------- Forwarded Message -------- > Subject: [keycloak-dev] changes to browser-based flows > Date: Fri, 16 Oct 2015 10:39:19 -0400 > From: Bill Burke > To: keycloak-dev at lists.jboss.org > > FYI: Not important if you are uninterested in design. > > Prior to my last commit if you hit the browser refresh button you would > either have had the authentication flow completely reset or received an > error page. Also, changing the local on some required actions pages > would end up in an error condition. > > So...To fix this I made some changes to browser based flows: > > * After any successful action processing (i.e. a form POST), the browser > is sent a 302 redirect to a "safer" page. If you are in the > authentication phase, then this redirect will be to > /authenticate?code={code}, registration /register?code={code}, reset > credentials /reset-credentials?code={code}, required actions > /required-action?code={code}. When these URIs are executed, Keycloak > will figure out where the user is in the flow and render things > appropriately. > * After authentication, the browser will be 302 redirected to > /required-action?code={code} > > The reason for these changes is to support when the user clicks the > browser refresh button. The refresh button will resubmit the previous > request. Prior to this change there were issues with this. For > example, previously, if there was a required action and you just logged > in via username and password, the URI in the browser would still point > to the username/password page even though the required action page was > being rendered. If the refresh button was hit, the previous username > password POST would be resent to the username/password page, Keycloak > would say "WTF are you doing?!?" and abort. There were similar issues > like this everywhere. > > Other things effected by this fix: > * required actions no longer change the ACTION_KEY or the > ClientSessionModel.getAction(). > * ClientSessionModel.getAction() will either be AUTHENTICATION, > REQUIRED_ACTIONS, EXECUTE_ACTIONS, LOGGED_OUT, or OAUTH_GRANT. > * After authentication, the flow manager will change the action from > AUTHENTICATION to REQUIRED_ACTIONS. > > Overall, this is less performant as there are additional HTTP redirect > requests being thrown in, but should provide a better user experience. > > > > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From erik.mulder at docdatapayments.com Mon Nov 16 07:26:01 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Mon, 16 Nov 2015 13:26:01 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B076@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B07A@DDPEX01.DDP.dcloud.local> <5644BC26.60602@gmail.com> Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B07C@DDPEX01.DDP.dcloud.local> @Christian: Actually in Hibernate 5 the Integrator does not offer the add entity feature anymore. It seems this 'feature' was actually an unintended side effect of the Integrator in Hibernate 4 (http://stackoverflow.com/questions/32918808/how-to-use-integrator-service-in-hibernate-5-for-adding-annotated-classes - "but the docs clearly point towards that not being intentional as this should be done at the time of initialization"). This complies with my finding that adding the entity classes at this 'late' stage skips some validation. So I guess I'll 'revert' my solution to the one I found previously, which adds the entity classes to the JPA persistence unit configuration. I'll investigate if that is Hibernate 4 and 5 compatible. @Stian: It seems like almost all questions have been answered and we can start to build the feature and contribute it to KeyCloak. Maybe you can provide some extra information / requirements / guidelines to us. (we did read the 'Hacking on Keycloak' page) - Is there a KeyCloak checkstyle configuration? - Do we have to create one entry in the KeyCloak JIRA for this or several under one epic? - Is there some peer review process? How does it work? - What are the requirements for the documentation? On 12/11/15 17:20, Christian Beikov wrote: You need different code for Hibernate 4 and 5 because the integrator interfaces are not compatible. The integrator is a service that is loaded through a java.util.ServiceLoader so I am not sure why you would need the property. Mit freundlichen Gr??en, ________________________________ Christian Beikov Am 12.11.2015 um 17:14 schrieb Erik Mulder: @Christian Beikov: Thanks for the hint on integrators! I was afraid I could not use integrators when working with the JPA 'route', but I found a way. There is a property EntityManagerFactoryBuilderImpl.INTEGRATOR_PROVIDER that you can use to supply custom integrators. This even works with the original 'pure' JPA call: Persistence.createEntityManagerFactory. In the integrator you can add annotated classes to the Hibernate Configuration. So the solution is still Hibernate only of course. The only downside is that there is less validation on the supplied classes, they are just added to the config directly. For instance I tested with adding Object.class as annotated class. This doesn't raise an exception and seems to be silently ignored by Hibernate. Not sure if it might result in problems during runtime though. Either way, normally this should be fine and with a little extra documentation on how these extra classes are handled I think using an integrator is the least intrusive thus best way to go. Agreed? Ok, List as collection type is fine too, indeed easier to use. Conceptually I like Set, because List seems to imply there is some kind of ordering involved which in this case there isn't. But that's just a minor matter of taste. By exotic ProviderLoader(Factory) I meant the org.keycloak.provider.ProviderLoader(Factory) that you can use to load Provider(Factory)'s from other locations than a file system based classpath. If that were to be a 'read once' kindof location (like some stream) there could be a problem. But with the integrator solution that shouldn't matter anymore, since it's just the Class object that we need. As for Hibernate 4/5, I'll try to make a solution that works for both Hibernate 4 and 5 in the same way. If that's not possible, I guess we could have separate code paths for 4 and 5, depending on the runtime version. I hope there is an easy way of figuring this out, maybe some static Hibernate class holding a version or so. A quick Google returns some useful results, so I'm sure we'll get that sorted if needed. End of the month is probably too soon indeed, so let's aim for 1.8. Should I (or someone else) create one (or several) JIRA tickets for this? On 12/11/15 14:32, Stian Thorgersen wrote: On 12 November 2015 at 13:58, Erik Mulder > wrote: On 11/11/15 13:54, Stian Thorgersen wrote: Would you be interested in contributing this feature? ATM we don't have anyone available that could work on it. A contribution would also need to include functional tests and documentation. Yes, I'd like to contribute this feature. I'm not sure about the timeline though. I hope to be able to do it as part of our current project, but I might have to use my spare time as well. Is there some kind of deadline to be included in a certain release version? We do a release every ~6 weeks. It's already a bit late for 1.7 (it's due end of the month) so would have to aim for 1.8 in either case (early January). If so I'm happy with going down the route of using the Hibernate specific classes. The remaining issue is figuring out how to deal with classloading. Looks like the following should work: * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider I've done this and it works fine, successfully providing the extra classes to the EntityManagerFactory build process in DefaultJpaConnectionProviderFactory. * JpaEntityProvider should have a single method "Class getEntities" Yes, only we need some kind of collection type, so you can provide multiple entity classes per provider. I guess you were intending this, considering the plural name 'getEntities'. I suggest either Collection> or Set> depending on what is most consistent with the rest of the system. Do you have a preference? Yup - List would be fine, that's what we tend to use as it's nicer to use than collection or set. * Implement org.hibernate.boot.registry.classloading.spi.ClassLoaderService - looks like this can just return null for everything except classForName where it would return the classes returned by the JpaEntityProvider implementations I see no way to interfere in the creation of the ClassLoaderService. The official way is using the BootstrapServiceRegistryBuilder, but with the JPA / EntityManagerFactoryBuilderImpl route this happens 'out of reach'. I did find another way that works just as well: you can provide a 'custom' classloader to the Bootstrap.getEntityManagerFactoryBuilder. We can define a classloader that will return the extra JpaEntityProvider classes if requested. Only tricky part here is that Hibernate not only calls loadClass on a classloader, but before that also getResource to get a URL with an InputStream to the class bytes. It uses that to scan for annotations with Jandex. I fixed this by forwarding that request to the ClassLoader of the JpaEntityProvider provided class (through Class.getClassLoader()). This works fine and shouldn't be problem for any drop in jars. I can imagine though that if you use some exotic ProviderLoader(Factory), you might somehow get in trouble if the class byte[] is not available anymore after class loading. But this is a problem with the way Hibernate works, not with the way we extend Hibernate in this case. So I think it's fine to have a warning about this in the documentation, since it will probably never be a real problem. If you consider this as a no-go, please let me know. Sounds OK, but not sure what you mean about exotic ProviderLoader(Factory) is that a Hibernate thing? Last question I have considers the Hibernate version of KeyCloak. Currently it's 4.3.10, are there any plans to upgrade to 5? The code related to classloading etc is refactored considerably in Hibernate 5. So it would be a shame to fully get it working for 4.3.10, only to have to upgrade soon after that. I didn't look into the details of Hibernate 5 and I think the solution we came up with should remain more or less intact, but you never know, so that's why I ask. We are soon moving to WildFly 10 which includes Hibernate 5, but we still need to support EAP 6.4 which includes Hibernate 4. At some point next year we will drop support for EAP 6.4 and move on to EAP 7. We either have to support both Hibernate 4 and 5 for a while, or we make it use the old approach on Hibernate 4 (so now custom entity class support on EAP 6.4) and the new approach on Hibernate 5. That would probably require some magic reflection code though. On 7 November 2015 at 23:39, Erik Mulder > wrote: On 06/11/15 14:46, Stian Thorgersen wrote: > We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? I was a little quick to state that with Hibernate you can add extra entity class names besides the one in persistence.xml, since I spotted a few answers on StackOverflow that said it could be done. But they resolve around classpath scanning or using a Spring managed Hibernate. Then I thought: 'if Spring can do it, I can do it too' so I investigated the Hibernate source code 'behind' Persistence.createEntityManagerFactory(unitName, properties). After some digging it turns out it's pretty simple to get extra class names in the configuration. See code sample below. The only problem is that Hibernate will only find classes that are part of the 'main' KeyCloak application, because of the way the Wildfly module system and ClassLoader strategy work. The debugger showed me Hibernate has these 3 class loaders available to look for classes: 1. ModuleClassLoader for Module "deployment.keycloak-server.war:main" from Service Module Loader 2. ModuleClassLoader for Module "org.hibernate:main" from local module loader 3. sun.misc.Launcher$AppClassLoader Number 1 has all other KeyCloak modules in it, so the entity classes from model-jpa will be found, but the wildfly-extensions module is missing, so entities in classes in a jar in the providers folder cannot be found. Now you guys obviously know a lot more about these internals, but as currently configured, it seems to me there is no way to let Hibernate 'see' these extra classes, since only the KeyCloak services module has a dependency on wildfly-extensions. So I think it boils down to these decisions: A. Do you accept a non-pure-JPA way of building the EntityManagerFactory that has some ties to the Hibernate internals? B. If A is no, than we're done. If yes, then you must find some way to get the extra configured classes 'into' Hibernate. You could get the wildfly-extensions module into scope of the Hibernate classloading. There are serveral ways to configure Hibernate classloading or you could flip some switches / dependencies in the module configuration. Another alternative is to create a separate 'dropfolder' besides themes and providers for JPA extensions, like 'models' or so and have that one be on the Hibernate classpath. But I don't know the exact design principles behind KeyCloak or the Wildfly module system. So maybe you have a better solution or maybe you conclude that this is 'not done' in terms of the architecture. Either way, I'd really appreciate some feedback on this and some thoughts on whether this could be a possible addition to KeyCloak in your eyes. Thanks, Erik Current JPA way. No way to 'interfere': emf = Persistence.createEntityManagerFactory(unitName, properties); Alternative Hibernate only way with adding extra entity class names: // Let Hibernate find and parse all 'persistence.xml' files found on the classpath. List persistenceUnits = PersistenceXmlParser.locatePersistenceUnits(properties); // Assume there is only one persistence unit found and that is the one we need. This can be made more robust by checking on the persistence unit name. ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = persistenceUnits.get(0); // Add extra class names. These could come from a 'JPA class name provider' SPI or something alike. persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity", "org.keycloak.models.jpa.entities.MerchantEntity"); // Let Hibernate create an EntityManagerFactory out of the (enriched) persistence unit configuration. emf = Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, properties).build(); On 6 November 2015 at 14:29, Erik Mulder <erik.mulder at docdatapayments.com> wrote: Thanks for pointing me explicitly to the SPI documentation. Of course that is exactly what I was looking for in my original question. I don't know how I overlooked this earlier! Probably I was not picking it up, because of almost a decade of developing on Spring projects, where this type of thing works differently. :-) I tried a quick test with a jar with one extra ProtocolMapper configured, put it in the providers folder and it worked like a charm! As for the JPA: We'll probably go with your suggestion of the separate EntityManagerFactory. Indeed there seems to be no way to 'programmatically extend' the list of entity classes in JPA besides editing or overwriting the persistence.xml. As you probably know it can be done in Hibernate, but I guess KeyCloak wants to stick to a generic JPA solution. That said, we might consider the Hibernate specific solution for our case, since being able to switch the JPA provider is not a requirement for us. And keeping the same connection/transaction is a lot easier in reasoning and debugging. We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? On 05/11/15 10:52, Stian Thorgersen wrote: The way to extend Keycloak is by implementing your own custom providers of the many SPIs we provide. Some SPIs are more stable (so marked as public) and others are not (so marked as private). If there are things that you want to customize that can't be done with an existing SPI then let us know and we may consider adding additional SPIs. On 4 November 2015 at 17:16, Erik Mulder <erik.mulder at docdatapayments.com> wrote: Thanks for your response! Indeed we already did a proof of concept where we added a custom mapper the way you described (didn't know it was 'protected' territory :). The question is: do we have to override the file 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper in the original project or is there another way where we don't need to touch the original sources and keep all our changes in a separate project? And how can we do it such that it stays easy to upgrade to newer KeyCloak releases? Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at the docs (http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) and examples for other provider (https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory) As for JPA: it would be easier to integrate with the existing JPA project. Again we are wondering whether to start modifying original sources (like persistence.xml) or try to 'externalize' our changes somehow and integrate them using existing 'hooks' in the system or maybe merge projects during build. Maybe there is no good answer to this and we'll always be having some manual merge pains when upgrading to new KeyCloak versions. We just wanted to check if there are preferred ways to add functionality with the least amount of impact on the original sources. I initially wanted the ability to add custom entities to the JpaConnectProvider, but couldn't find a way to define entities programatically with JPA. To add your own persistence.xml you would have to define your own implementation of JpaConnectionProvider and change what is loaded by default (connectionsJpa/provider attribute in keycloak-server.json). Alternative, which is cleaner, but you end up with separate connection/transaction, is to create your own EntityManagerFactory. If it's only used by one provider (for example a custom UserFederationProvider) there's no need to add a connect provider (that's just a way to share one EntityManagerFactory between multiple providers) and you can just create it in the MyUserFederationProviderFactory. On 04/11/15 15:30, Bill Burke wrote: > Custom mappers should be possible. I didn't document it as I wasn't > sure if we wanted to make the SPI public. Custom mappers should just > follow the Provider SPI and they will be picked up. If you see the > META-INF/services/... file in the resources directory of the "services" > or "broker" modules you'll see how to set this up. > > As for extending the JPA datamodel, what you could do is write a new JPA > Connections Provider and plug that in. See connections/jpa. I'm not > sure how you would handle the liquibase db migration. > > On 11/4/2015 6:03 AM, Erik Mulder wrote: >> Hi everybody, >> >> Quick intro: I?m part of a development team in The Netherlands that is >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >> this and will use OpenID Connect to secure our REST services. It?s a >> great product and seems to be the only one having both support for all >> kinds of security standards and a model and GUI for users and roles. >> Thanks for creating it! J >> >> (if this should be asked instead on the users mailing list, please >> correct me and I?ll post it there) >> >> So far, so good, but we have some extra requirements that do not fit >> into the base KeyCloak data model. See below for details if you?re >> interested. My question is: what is the preferred way / best practice to >> extend the functionality of KeyCloak while keeping the impact on the >> original sources to a minimum? Of course we could just fork the most >> recent version and start hacking away, but we?d like to be able to >> upgrade to newer versions of KeyCloak without too much hassle. >> Possibilities that we?ve come up with so far: >> >> 1.Create completely separate modules that will extend the functionality >> the way we need. >> >> 2.Fork on Github, apply custom changes, and try to merge in updates from >> the master / release branches / tags >> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >> as Truezip >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >> manipulate zip files by adding/removing/replacing or Shade >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple >> jars to 1 'uber-jar' containing the contents of both and when >> overlapping decide on conflicts through configuration. >> >> Of course number 1 is preferred, but I do not see how to add custom >> mappers or JPA entities without making changes in the original module >> files. The other options seem like valid alternatives, but maybe there >> is better / standard way to do this. So any help / insight / shared >> experience on this is much appreciated, thanks! >> >> Kind regards, >> >> Erik Mulder >> >> Senior Software Engineer >> >> Docdata Payments ? NL >> >> P.S. Details on why we want to extend the KeyCloak data model: (any >> feedback on the contents of this P.S. is also welcome!) >> >> Our clients are merchants that have several webshops. We manage their >> online payments (shopping cart checkout). We want to be able to let a >> merchant manage their own users and let a user have different roles for >> different webshops within the same merchant. The overall possible roles >> are fixed though, no specific roles per merchant. We could create a >> separate realm for every merchant, but then we need to duplicate all >> roles every time. Furthermore, in KeyCloak there is no concept of a role >> within a certain context. This is very understandable, since every >> situation has it?s own requirements. We did a proof of concept by adding >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >> and adding a custom mapper that can put this information on the Access >> token. Worked like a charm! But it does need some changes in the >> KeyCloak modules and sources to work, hence the question above. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151116/a16946dd/attachment-0001.html From sthorger at redhat.com Mon Nov 16 07:41:32 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 16 Nov 2015 13:41:32 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml In-Reply-To: <9A5619B792BBA041AE094585791BB71C0137B668B07C@DDPEX01.DDP.dcloud.local> References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B076@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B07A@DDPEX01.DDP.dcloud.local> <5644BC26.60602@gmail.com> <9A5619B792BBA041AE094585791BB71C0137B668B07C@DDPEX01.DDP.dcloud.local> Message-ID: On 16 November 2015 at 13:26, Erik Mulder wrote: > *@Christian:* Actually in Hibernate 5 the Integrator does not offer the > add entity feature anymore. It seems this 'feature' was actually an > unintended side effect of the Integrator in Hibernate 4 ( > http://stackoverflow.com/questions/32918808/how-to-use-integrator-service-in-hibernate-5-for-adding-annotated-classes > - *"but the docs clearly point towards that not being intentional as this > should be done at the time of initialization"*). This complies with my > finding that adding the entity classes at this 'late' stage skips some > validation. So I guess I'll 'revert' my solution to the one I found > previously, which adds the entity classes to the JPA persistence unit > configuration. I'll investigate if that is Hibernate 4 and 5 compatible. > > *@Stian: *It seems like almost all questions have been answered and we > can start to build the feature and contribute it to KeyCloak. Maybe you can > provide some extra information / requirements / guidelines to us. (we did > read the 'Hacking on Keycloak' page) > - Is there a KeyCloak checkstyle configuration? > No, we have not introduced one yet. However, we'd like code to follow code style from WildFly. You can get IDE configuration from here https://github.com/wildfly/wildfly-core/tree/master/ide-configs > - Do we have to create one entry in the KeyCloak JIRA for this or several > under one epic? > One is sufficient > - Is there some peer review process? How does it work? > Step one is to send an email to Keycloak Dev mailing list about what you are doing. This is obviously already done. Second stage is when you've prepared a PR one of the core developers will review it before merging. > - What are the requirements for the documentation? > No hard requirements, but in general for a new feature to be added it needs to be mentioned on the documentation. For this particular feature I think adding a section to http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html would be good. > > > > > On 12/11/15 17:20, Christian Beikov wrote: > > You need different code for Hibernate 4 and 5 because the integrator > interfaces are not compatible. > The integrator is a service that is loaded through a > java.util.ServiceLoader so I am not sure why you would need the property. > > Mit freundlichen Gr??en, > ------------------------------ > *Christian Beikov* > Am 12.11.2015 um 17:14 schrieb Erik Mulder: > > *@Christian Beikov*: Thanks for the hint on integrators! > > I was afraid I could not use integrators when working with the JPA > 'route', but I found a way. There is a property EntityManagerFactoryBuilderImpl.INTEGRATOR_PROVIDER > that you can use to supply custom integrators. This even works with the > original 'pure' JPA call: Persistence.createEntityManagerFactory. In the > integrator you can add annotated classes to the Hibernate Configuration. So > the solution is still Hibernate only of course. > > The only downside is that there is less validation on the supplied > classes, they are just added to the config directly. For instance I tested > with adding Object.class as annotated class. This doesn't raise an > exception and seems to be silently ignored by Hibernate. Not sure if it > might result in problems during runtime though. Either way, normally this > should be fine and with a little extra documentation on how these extra > classes are handled I think using an integrator is the least intrusive thus > best way to go. Agreed? > > Ok, List as collection type is fine too, indeed easier to use. > Conceptually I like Set, because List seems to imply there is some kind of > ordering involved which in this case there isn't. But that's just a minor > matter of taste. > > By exotic ProviderLoader(Factory) I meant the > org.keycloak.provider.ProviderLoader(Factory) that you can use to load > Provider(Factory)'s from other locations than a file system based > classpath. If that were to be a 'read once' kindof location (like some > stream) there could be a problem. But with the integrator solution that > shouldn't matter anymore, since it's just the Class object that we need. > > As for Hibernate 4/5, I'll try to make a solution that works for both > Hibernate 4 and 5 in the same way. If that's not possible, I guess we could > have separate code paths for 4 and 5, depending on the runtime version. I > hope there is an easy way of figuring this out, maybe some static Hibernate > class holding a version or so. A quick Google returns some useful results, > so I'm sure we'll get that sorted if needed. > > End of the month is probably too soon indeed, so let's aim for 1.8. Should > I (or someone else) create one (or several) JIRA tickets for this? > > > On 12/11/15 14:32, Stian Thorgersen wrote: > > > > On 12 November 2015 at 13:58, Erik Mulder > wrote: > >> On 11/11/15 13:54, Stian Thorgersen wrote: >> >> Would you be interested in contributing this feature? ATM we don't have >> anyone available that could work on it. A contribution would also need to >> include functional tests and documentation. >> >> >> Yes, I'd like to contribute this feature. I'm not sure about the timeline >> though. I hope to be able to do it as part of our current project, but I >> might have to use my spare time as well. Is there some kind of deadline to >> be included in a certain release version? >> > > We do a release every ~6 weeks. It's already a bit late for 1.7 (it's due > end of the month) so would have to aim for 1.8 in either case (early > January). > > >> >> >> If so I'm happy with going down the route of using the Hibernate specific >> classes. The remaining issue is figuring out how to deal with classloading. >> >> Looks like the following should work: >> >> * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider >> >> >> I've done this and it works fine, successfully providing the extra >> classes to the EntityManagerFactory build process in >> DefaultJpaConnectionProviderFactory. >> >> * JpaEntityProvider should have a single method "Class getEntities" >> >> >> Yes, only we need some kind of collection type, so you can provide >> multiple entity classes per provider. I guess you were intending this, >> considering the plural name 'getEntities'. I suggest either >> Collection> or Set> depending on what is most >> consistent with the rest of the system. Do you have a preference? >> > > Yup - List would be fine, that's what we tend to use as it's nicer to use > than collection or set. > > >> >> >> * Implement org.hibernate.boot.registry.classloading.spi.ClassLoaderService >> - looks like this can just return null for everything except classForName >> where it would return the classes returned by the JpaEntityProvider >> implementations >> >> >> I see no way to interfere in the creation of the ClassLoaderService. The >> official way is using the BootstrapServiceRegistryBuilder, but with the >> JPA / EntityManagerFactoryBuilderImpl route this happens 'out of reach'. >> I did find another way that works just as well: you can provide a 'custom' >> classloader to the Bootstrap.getEntityManagerFactoryBuilder. We can >> define a classloader that will return the extra JpaEntityProvider classes >> if requested. Only tricky part here is that Hibernate not only calls >> loadClass on a classloader, but before that also getResource to get a URL >> with an InputStream to the class bytes. It uses that to scan for >> annotations with Jandex. I fixed this by forwarding that request to the >> ClassLoader of the JpaEntityProvider provided class (through >> Class.getClassLoader()). This works fine and shouldn't be problem for >> any drop in jars. I can imagine though that if you use some exotic >> ProviderLoader(Factory), you might somehow get in trouble if the class >> byte[] is not available anymore after class loading. But this is a problem >> with the way Hibernate works, not with the way we extend Hibernate in this >> case. So I think it's fine to have a warning about this in the >> documentation, since it will probably never be a real problem. If you >> consider this as a no-go, please let me know. >> > > Sounds OK, but not sure what you mean about exotic ProviderLoader(Factory) > is that a Hibernate thing? > > >> >> >> Last question I have considers the Hibernate version of KeyCloak. >> Currently it's 4.3.10, are there any plans to upgrade to 5? The code >> related to classloading etc is refactored considerably in Hibernate 5. So >> it would be a shame to fully get it working for 4.3.10, only to have to >> upgrade soon after that. I didn't look into the details of Hibernate 5 and >> I think the solution we came up with should remain more or less intact, but >> you never know, so that's why I ask. >> > > We are soon moving to WildFly 10 which includes Hibernate 5, but we still > need to support EAP 6.4 which includes Hibernate 4. At some point next year > we will drop support for EAP 6.4 and move on to EAP 7. > > We either have to support both Hibernate 4 and 5 for a while, or we make > it use the old approach on Hibernate 4 (so now custom entity class support > on EAP 6.4) and the new approach on Hibernate 5. That would probably > require some magic reflection code though. > > >> >> >> >> On 7 November 2015 at 23:39, Erik Mulder > > wrote: >> >>> *On 06/11/15 14:46, Stian Thorgersen wrote:* >>> *> We could use Hibernate directly to boostrap as long as it can return >>> an EntityManager. Do you know if that's possible?* >>> >>> I was a little quick to state that with Hibernate you can add extra >>> entity class names besides the one in persistence.xml, since I spotted a >>> few answers on StackOverflow that said it could be done. But they resolve >>> around classpath scanning or using a Spring managed Hibernate. Then I >>> thought: 'if Spring can do it, I can do it too' so I investigated the >>> Hibernate source code 'behind' >>> Persistence.createEntityManagerFactory(unitName, properties). After >>> some digging it turns out it's pretty simple to get extra class names in >>> the configuration. See code sample below. >>> >>> The only problem is that Hibernate will only find classes that are part >>> of the 'main' KeyCloak application, because of the way the Wildfly module >>> system and ClassLoader strategy work. The debugger showed me Hibernate has >>> these 3 class loaders available to look for classes: >>> 1. ModuleClassLoader for Module "deployment.keycloak-server.war:main" >>> from Service Module Loader >>> 2. ModuleClassLoader for Module "org.hibernate:main" from local module >>> loader >>> 3. sun.misc.Launcher$AppClassLoader >>> >>> Number 1 has all other KeyCloak modules in it, so the entity classes >>> from model-jpa will be found, but the wildfly-extensions module is missing, >>> so entities in classes in a jar in the providers folder cannot be found. >>> Now you guys obviously know a lot more about these internals, but as >>> currently configured, it seems to me there is no way to let Hibernate 'see' >>> these extra classes, since only the KeyCloak services module has a >>> dependency on wildfly-extensions. >>> >>> So I think it boils down to these decisions: >>> A. Do you accept a non-pure-JPA way of building the EntityManagerFactory >>> that has some ties to the Hibernate internals? >>> B. If A is no, than we're done. If yes, then you must find some way to >>> get the extra configured classes 'into' Hibernate. You could get the >>> wildfly-extensions module into scope of the Hibernate classloading. There >>> are serveral ways to configure Hibernate classloading or you could flip >>> some switches / dependencies in the module configuration. Another >>> alternative is to create a separate 'dropfolder' besides themes and >>> providers for JPA extensions, like 'models' or so and have that one be on >>> the Hibernate classpath. But I don't know the exact design principles >>> behind KeyCloak or the Wildfly module system. So maybe you have a better >>> solution or maybe you conclude that this is 'not done' in terms of the >>> architecture. >>> >>> Either way, I'd really appreciate some feedback on this and some >>> thoughts on whether this could be a possible addition to KeyCloak in your >>> eyes. >>> >>> Thanks, Erik >>> >>> >>> Current JPA way. No way to 'interfere': >>> emf = Persistence.createEntityManagerFactory(unitName, properties); >>> >>> Alternative Hibernate only way with adding extra entity class names: >>> // Let Hibernate find and parse all 'persistence.xml' files found on the >>> classpath. >>> List persistenceUnits = >>> PersistenceXmlParser.locatePersistenceUnits(properties); >>> // Assume there is only one persistence unit found and that is the one >>> we need. This can be made more robust by checking on the persistence unit >>> name. >>> ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = >>> persistenceUnits.get(0); >>> // Add extra class names. These could come from a 'JPA class name >>> provider' SPI or something alike. >>> >>> persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity", >>> "org.keycloak.models.jpa.entities.MerchantEntity"); >>> // Let Hibernate create an EntityManagerFactory out of the (enriched) >>> persistence unit configuration. >>> emf = >>> Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, >>> properties).build(); >>> >>> >>> >>> >>> On 6 November 2015 at 14:29, Erik Mulder < >>> erik.mulder at docdatapayments.com> wrote: >>> >>>> Thanks for pointing me explicitly to the SPI documentation. Of course >>>> that is exactly what I was looking for in my original question. I don't >>>> know how I overlooked this earlier! Probably I was not picking it up, >>>> because of almost a decade of developing on Spring projects, where this >>>> type of thing works differently. :-) >>>> >>>> I tried a quick test with a jar with one extra ProtocolMapper >>>> configured, put it in the providers folder and it worked like a charm! >>>> >>>> As for the JPA: We'll probably go with your suggestion of the separate >>>> EntityManagerFactory. Indeed there seems to be no way to 'programmatically >>>> extend' the list of entity classes in JPA besides editing or overwriting >>>> the persistence.xml. As you probably know it can be done in Hibernate, but >>>> I guess KeyCloak wants to stick to a generic JPA solution. That said, we >>>> might consider the Hibernate specific solution for our case, since being >>>> able to switch the JPA provider is not a requirement for us. And keeping >>>> the same connection/transaction is a lot easier in reasoning and debugging. >>>> >>> >>> We could use Hibernate directly to boostrap as long as it can return an >>> EntityManager. Do you know if that's possible? >>> >>> >>>> >>>> >>>> >>>> On 05/11/15 10:52, Stian Thorgersen wrote: >>>> >>>> The way to extend Keycloak is by implementing your own custom providers >>>> of the many SPIs we provide. Some SPIs are more stable (so marked as >>>> public) and others are not (so marked as private). If there are things that >>>> you want to customize that can't be done with an existing SPI then let us >>>> know and we may consider adding additional SPIs. >>>> >>>> On 4 November 2015 at 17:16, Erik Mulder < >>>> erik.mulder at docdatapayments.com> >>>> wrote: >>>> >>>>> Thanks for your response! >>>>> >>>>> Indeed we already did a proof of concept where we added a custom mapper >>>>> the way you described (didn't know it was 'protected' territory :). The >>>>> question is: do we have to override the file >>>>> 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper >>>>> in the original project or is there another way where we don't need to >>>>> touch the original sources and keep all our changes in a separate >>>>> project? And how can we do it such that it stays easy to upgrade to >>>>> newer KeyCloak releases? >>>>> >>>> >>>> Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look >>>> at the docs ( >>>> >>>> http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) >>>> and examples for other provider ( >>>> >>>> https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory >>>> ) >>>> >>>> >>>>> >>>>> As for JPA: it would be easier to integrate with the existing JPA >>>>> project. Again we are wondering whether to start modifying original >>>>> sources (like persistence.xml) or try to 'externalize' our changes >>>>> somehow and integrate them using existing 'hooks' in the system or >>>>> maybe >>>>> merge projects during build. >>>>> >>>>> Maybe there is no good answer to this and we'll always be having some >>>>> manual merge pains when upgrading to new KeyCloak versions. We just >>>>> wanted to check if there are preferred ways to add functionality with >>>>> the least amount of impact on the original sources. >>>>> >>>> >>>> I initially wanted the ability to add custom entities to the >>>> JpaConnectProvider, but couldn't find a way to define entities >>>> programatically with JPA. To add your own persistence.xml you would have to >>>> define your own implementation of JpaConnectionProvider and change what is >>>> loaded by default (connectionsJpa/provider attribute in >>>> keycloak-server.json). >>>> >>>> Alternative, which is cleaner, but you end up with separate >>>> connection/transaction, is to create your own EntityManagerFactory. If it's >>>> only used by one provider (for example a custom UserFederationProvider) >>>> there's no need to add a connect provider (that's just a way to share one >>>> EntityManagerFactory between multiple providers) and you can just create it >>>> in the MyUserFederationProviderFactory. >>>> >>>> >>>>> >>>>> >>>>> On 04/11/15 15:30, Bill Burke wrote: >>>>> > Custom mappers should be possible. I didn't document it as I wasn't >>>>> > sure if we wanted to make the SPI public. Custom mappers should just >>>>> > follow the Provider SPI and they will be picked up. If you see the >>>>> > META-INF/services/... file in the resources directory of the >>>>> "services" >>>>> > or "broker" modules you'll see how to set this up. >>>>> > >>>>> > As for extending the JPA datamodel, what you could do is write a new >>>>> JPA >>>>> > Connections Provider and plug that in. See connections/jpa. I'm not >>>>> > sure how you would handle the liquibase db migration. >>>>> > >>>>> > On 11/4/2015 6:03 AM, Erik Mulder wrote: >>>>> >> Hi everybody, >>>>> >> >>>>> >> Quick intro: I?m part of a development team in The Netherlands that >>>>> is >>>>> >> building a company-wide SSO solution. We?ve chosen KeyCloak to >>>>> realize >>>>> >> this and will use OpenID Connect to secure our REST services. It?s a >>>>> >> great product and seems to be the only one having both support for >>>>> all >>>>> >> kinds of security standards and a model and GUI for users and roles. >>>>> >> Thanks for creating it! J >>>>> >> >>>>> >> (if this should be asked instead on the users mailing list, please >>>>> >> correct me and I?ll post it there) >>>>> >> >>>>> >> So far, so good, but we have some extra requirements that do not fit >>>>> >> into the base KeyCloak data model. See below for details if you?re >>>>> >> interested. My question is: what is the preferred way / best >>>>> practice to >>>>> >> extend the functionality of KeyCloak while keeping the impact on the >>>>> >> original sources to a minimum? Of course we could just fork the most >>>>> >> recent version and start hacking away, but we?d like to be able to >>>>> >> upgrade to newer versions of KeyCloak without too much hassle. >>>>> >> Possibilities that we?ve come up with so far: >>>>> >> >>>>> >> 1.Create completely separate modules that will extend the >>>>> functionality >>>>> >> the way we need. >>>>> >> >>>>> >> 2.Fork on Github, apply custom changes, and try to merge in updates >>>>> from >>>>> >> the master / release branches / tags >>>>> >> >>>>> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, >>>>> such >>>>> >> as Truezip >>>>> >> ( >>>>> http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >>>>> >> manipulate zip files by adding/removing/replacing or Shade >>>>> >> ( >>>>> http://maven.apache.org/plugins/maven-shade-plugin/) - combine >>>>> multiple >>>>> >> jars to 1 'uber-jar' containing the contents of both and when >>>>> >> overlapping decide on conflicts through configuration. >>>>> >> >>>>> >> Of course number 1 is preferred, but I do not see how to add custom >>>>> >> mappers or JPA entities without making changes in the original >>>>> module >>>>> >> files. The other options seem like valid alternatives, but maybe >>>>> there >>>>> >> is better / standard way to do this. So any help / insight / shared >>>>> >> experience on this is much appreciated, thanks! >>>>> >> >>>>> >> Kind regards, >>>>> >> >>>>> >> Erik Mulder >>>>> >> >>>>> >> Senior Software Engineer >>>>> >> >>>>> >> Docdata Payments ? NL >>>>> >> >>>>> >> P.S. Details on why we want to extend the KeyCloak data model: (any >>>>> >> feedback on the contents of this P.S. is also welcome!) >>>>> >> >>>>> >> Our clients are merchants that have several webshops. We manage >>>>> their >>>>> >> online payments (shopping cart checkout). We want to be able to let >>>>> a >>>>> >> merchant manage their own users and let a user have different roles >>>>> for >>>>> >> different webshops within the same merchant. The overall possible >>>>> roles >>>>> >> are fixed though, no specific roles per merchant. We could create a >>>>> >> separate realm for every merchant, but then we need to duplicate all >>>>> >> roles every time. Furthermore, in KeyCloak there is no concept of a >>>>> role >>>>> >> within a certain context. This is very understandable, since every >>>>> >> situation has it?s own requirements. We did a proof of concept by >>>>> adding >>>>> >> tables and entities for Merchant, UserMerchant, UserMerchantRole >>>>> etc. >>>>> >> and adding a custom mapper that can put this information on the >>>>> Access >>>>> >> token. Worked like a charm! But it does need some changes in the >>>>> >> KeyCloak modules and sources to work, hence the question above. >>>>> >> >>>>> >> >>>>> >> >>>>> >> _______________________________________________ >>>>> >> keycloak-dev mailing list >>>>> >> >>>>> keycloak-dev at lists.jboss.org >>>>> >> >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >> >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>> >>>> >>>> >>> >>> >> >> > > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151116/953a3091/attachment-0001.html From sthorger at redhat.com Mon Nov 16 08:54:58 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 16 Nov 2015 14:54:58 +0100 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one Message-ID: We have both "id" and "client-id" for clients in Keycloak at the moment. This seems unnecessary and complex. The model can retrieve clients on either value. In token endpoints the "client-id" is used. In admin endpoints the "id" is used. Also, in most cases it would be simpler for users to just have a generated id than having to come up with one themselves. The id doesn't have to be human readable either as we have name for that. OpenID Connect expects "client-id" to be generated by the IdP and can't be changed once created. I propose we remove "client-id" and only keep id. For migration of existing clients we would set the "id" value to the current value of "client-id". This would require no changes to adapter configs. When creating new clients from the admin console we would not allow setting the "client-id", instead just display it after the client was created. When importing clients it would be possible to set the id (and for backwards compatibility we would set "id" equal to the "client-id" field. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151116/52c5e137/attachment.html From bburke at redhat.com Mon Nov 16 09:21:21 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 16 Nov 2015 09:21:21 -0500 Subject: [keycloak-dev] Why do we use Beans in Freemarker templates? Message-ID: <5649E661.8050106@redhat.com> Why do we use Beans in Freemarker templates? Most of the information can be obtained from Model objects. Any reason Model objects can't be accessed directly from .ftl files? -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Mon Nov 16 09:27:31 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 16 Nov 2015 15:27:31 +0100 Subject: [keycloak-dev] Why do we use Beans in Freemarker templates? In-Reply-To: <5649E661.8050106@redhat.com> References: <5649E661.8050106@redhat.com> Message-ID: It comes from the olden days when we were doing a SaaS solution as we wanted to reduce what was possible from templates. It makes less sense now, although can be useful in the cases when you need some logic or formatting, or the values aren't directly available from the models. On 16 November 2015 at 15:21, Bill Burke wrote: > Why do we use Beans in Freemarker templates? Most of the information > can be obtained from Model objects. Any reason Model objects can't be > accessed directly from .ftl files? > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151116/86fac8be/attachment.html From bburke at redhat.com Mon Nov 16 09:29:45 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 16 Nov 2015 09:29:45 -0500 Subject: [keycloak-dev] Why do we use Beans in Freemarker templates? In-Reply-To: References: <5649E661.8050106@redhat.com> Message-ID: <5649E859.9020408@redhat.com> Needs to be cleaned up eventually. Just a pain for documentation since the Auth SPI exposes it. On 11/16/2015 9:27 AM, Stian Thorgersen wrote: > It comes from the olden days when we were doing a SaaS solution as we > wanted to reduce what was possible from templates. > > It makes less sense now, although can be useful in the cases when you > need some logic or formatting, or the values aren't directly available > from the models. > > On 16 November 2015 at 15:21, Bill Burke > wrote: > > Why do we use Beans in Freemarker templates? Most of the information > can be obtained from Model objects. Any reason Model objects can't be > accessed directly from .ftl files? > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Mon Nov 16 09:40:06 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 16 Nov 2015 15:40:06 +0100 Subject: [keycloak-dev] Why do we use Beans in Freemarker templates? In-Reply-To: <5649E859.9020408@redhat.com> References: <5649E661.8050106@redhat.com> <5649E859.9020408@redhat.com> Message-ID: +1 It's a mess and very outdated On 16 November 2015 at 15:29, Bill Burke wrote: > Needs to be cleaned up eventually. Just a pain for documentation since > the Auth SPI exposes it. > > On 11/16/2015 9:27 AM, Stian Thorgersen wrote: > >> It comes from the olden days when we were doing a SaaS solution as we >> wanted to reduce what was possible from templates. >> >> It makes less sense now, although can be useful in the cases when you >> need some logic or formatting, or the values aren't directly available >> from the models. >> >> On 16 November 2015 at 15:21, Bill Burke > > wrote: >> >> Why do we use Beans in Freemarker templates? Most of the information >> can be obtained from Model objects. Any reason Model objects can't be >> accessed directly from .ftl files? >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151116/dc56ffd9/attachment.html From erik.mulder at docdatapayments.com Mon Nov 16 09:41:56 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Mon, 16 Nov 2015 15:41:56 +0100 Subject: [keycloak-dev] Preferred way to make KeyCloak custom changes - Allow for extra entities in Hibernate besides persistence.xml References: <9A5619B792BBA041AE094585791BB71C0137B668B063@DDPEX01.DDP.dcloud.local> <563A168A.2010402@redhat.com> <9A5619B792BBA041AE094585791BB71C0137B668B065@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B067@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B06D@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B076@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668B07A@DDPEX01.DDP.dcloud.local> <5644BC26.60602@gmail.com> <9A5619B792BBA041AE094585791BB71C0137B668B07C@DDPEX01.DDP.dcloud.local> Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B07E@DDPEX01.DDP.dcloud.local> Ok, all clear! Thanks for the discussion/information so far. We'll implement this feature in one of our coming sprints. We aim to be finished in time for 1.8. On 16/11/15 13:41, Stian Thorgersen wrote: On 16 November 2015 at 13:26, Erik Mulder > wrote: @Christian: Actually in Hibernate 5 the Integrator does not offer the add entity feature anymore. It seems this 'feature' was actually an unintended side effect of the Integrator in Hibernate 4 (http://stackoverflow.com/questions/32918808/how-to-use-integrator-service-in-hibernate-5-for-adding-annotated-classes - "but the docs clearly point towards that not being intentional as this should be done at the time of initialization"). This complies with my finding that adding the entity classes at this 'late' stage skips some validation. So I guess I'll 'revert' my solution to the one I found previously, which adds the entity classes to the JPA persistence unit configuration. I'll investigate if that is Hibernate 4 and 5 compatible. @Stian: It seems like almost all questions have been answered and we can start to build the feature and contribute it to KeyCloak. Maybe you can provide some extra information / requirements / guidelines to us. (we did read the 'Hacking on Keycloak' page) - Is there a KeyCloak checkstyle configuration? No, we have not introduced one yet. However, we'd like code to follow code style from WildFly. You can get IDE configuration from here https://github.com/wildfly/wildfly-core/tree/master/ide-configs - Do we have to create one entry in the KeyCloak JIRA for this or several under one epic? One is sufficient - Is there some peer review process? How does it work? Step one is to send an email to Keycloak Dev mailing list about what you are doing. This is obviously already done. Second stage is when you've prepared a PR one of the core developers will review it before merging. - What are the requirements for the documentation? No hard requirements, but in general for a new feature to be added it needs to be mentioned on the documentation. For this particular feature I think adding a section to http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html would be good. On 12/11/15 17:20, Christian Beikov wrote: You need different code for Hibernate 4 and 5 because the integrator interfaces are not compatible. The integrator is a service that is loaded through a java.util.ServiceLoader so I am not sure why you would need the property. Mit freundlichen Gr??en, ________________________________ Christian Beikov Am 12.11.2015 um 17:14 schrieb Erik Mulder: @Christian Beikov: Thanks for the hint on integrators! I was afraid I could not use integrators when working with the JPA 'route', but I found a way. There is a property EntityManagerFactoryBuilderImpl.INTEGRATOR_PROVIDER that you can use to supply custom integrators. This even works with the original 'pure' JPA call: Persistence.createEntityManagerFactory. In the integrator you can add annotated classes to the Hibernate Configuration. So the solution is still Hibernate only of course. The only downside is that there is less validation on the supplied classes, they are just added to the config directly. For instance I tested with adding Object.class as annotated class. This doesn't raise an exception and seems to be silently ignored by Hibernate. Not sure if it might result in problems during runtime though. Either way, normally this should be fine and with a little extra documentation on how these extra classes are handled I think using an integrator is the least intrusive thus best way to go. Agreed? Ok, List as collection type is fine too, indeed easier to use. Conceptually I like Set, because List seems to imply there is some kind of ordering involved which in this case there isn't. But that's just a minor matter of taste. By exotic ProviderLoader(Factory) I meant the org.keycloak.provider.ProviderLoader(Factory) that you can use to load Provider(Factory)'s from other locations than a file system based classpath. If that were to be a 'read once' kindof location (like some stream) there could be a problem. But with the integrator solution that shouldn't matter anymore, since it's just the Class object that we need. As for Hibernate 4/5, I'll try to make a solution that works for both Hibernate 4 and 5 in the same way. If that's not possible, I guess we could have separate code paths for 4 and 5, depending on the runtime version. I hope there is an easy way of figuring this out, maybe some static Hibernate class holding a version or so. A quick Google returns some useful results, so I'm sure we'll get that sorted if needed. End of the month is probably too soon indeed, so let's aim for 1.8. Should I (or someone else) create one (or several) JIRA tickets for this? On 12/11/15 14:32, Stian Thorgersen wrote: On 12 November 2015 at 13:58, Erik Mulder <erik.mulder at docdatapayments.com> wrote: On 11/11/15 13:54, Stian Thorgersen wrote: Would you be interested in contributing this feature? ATM we don't have anyone available that could work on it. A contribution would also need to include functional tests and documentation. Yes, I'd like to contribute this feature. I'm not sure about the timeline though. I hope to be able to do it as part of our current project, but I might have to use my spare time as well. Is there some kind of deadline to be included in a certain release version? We do a release every ~6 weeks. It's already a bit late for 1.7 (it's due end of the month) so would have to aim for 1.8 in either case (early January). If so I'm happy with going down the route of using the Hibernate specific classes. The remaining issue is figuring out how to deal with classloading. Looks like the following should work: * Add JpaEntitySPI, JpaEntityProviderFactory and JpaEntityProvider I've done this and it works fine, successfully providing the extra classes to the EntityManagerFactory build process in DefaultJpaConnectionProviderFactory. * JpaEntityProvider should have a single method "Class getEntities" Yes, only we need some kind of collection type, so you can provide multiple entity classes per provider. I guess you were intending this, considering the plural name 'getEntities'. I suggest either Collection> or Set> depending on what is most consistent with the rest of the system. Do you have a preference? Yup - List would be fine, that's what we tend to use as it's nicer to use than collection or set. * Implement org.hibernate.boot.registry.classloading.spi.ClassLoaderService - looks like this can just return null for everything except classForName where it would return the classes returned by the JpaEntityProvider implementations I see no way to interfere in the creation of the ClassLoaderService. The official way is using the BootstrapServiceRegistryBuilder, but with the JPA / EntityManagerFactoryBuilderImpl route this happens 'out of reach'. I did find another way that works just as well: you can provide a 'custom' classloader to the Bootstrap.getEntityManagerFactoryBuilder. We can define a classloader that will return the extra JpaEntityProvider classes if requested. Only tricky part here is that Hibernate not only calls loadClass on a classloader, but before that also getResource to get a URL with an InputStream to the class bytes. It uses that to scan for annotations with Jandex. I fixed this by forwarding that request to the ClassLoader of the JpaEntityProvider provided class (through Class.getClassLoader()). This works fine and shouldn't be problem for any drop in jars. I can imagine though that if you use some exotic ProviderLoader(Factory), you might somehow get in trouble if the class byte[] is not available anymore after class loading. But this is a problem with the way Hibernate works, not with the way we extend Hibernate in this case. So I think it's fine to have a warning about this in the documentation, since it will probably never be a real problem. If you consider this as a no-go, please let me know. Sounds OK, but not sure what you mean about exotic ProviderLoader(Factory) is that a Hibernate thing? Last question I have considers the Hibernate version of KeyCloak. Currently it's 4.3.10, are there any plans to upgrade to 5? The code related to classloading etc is refactored considerably in Hibernate 5. So it would be a shame to fully get it working for 4.3.10, only to have to upgrade soon after that. I didn't look into the details of Hibernate 5 and I think the solution we came up with should remain more or less intact, but you never know, so that's why I ask. We are soon moving to WildFly 10 which includes Hibernate 5, but we still need to support EAP 6.4 which includes Hibernate 4. At some point next year we will drop support for EAP 6.4 and move on to EAP 7. We either have to support both Hibernate 4 and 5 for a while, or we make it use the old approach on Hibernate 4 (so now custom entity class support on EAP 6.4) and the new approach on Hibernate 5. That would probably require some magic reflection code though. On 7 November 2015 at 23:39, Erik Mulder <erik.mulder at docdatapayments.com> wrote: On 06/11/15 14:46, Stian Thorgersen wrote: > We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? I was a little quick to state that with Hibernate you can add extra entity class names besides the one in persistence.xml, since I spotted a few answers on StackOverflow that said it could be done. But they resolve around classpath scanning or using a Spring managed Hibernate. Then I thought: 'if Spring can do it, I can do it too' so I investigated the Hibernate source code 'behind' Persistence.createEntityManagerFactory(unitName, properties). After some digging it turns out it's pretty simple to get extra class names in the configuration. See code sample below. The only problem is that Hibernate will only find classes that are part of the 'main' KeyCloak application, because of the way the Wildfly module system and ClassLoader strategy work. The debugger showed me Hibernate has these 3 class loaders available to look for classes: 1. ModuleClassLoader for Module "deployment.keycloak-server.war:main" from Service Module Loader 2. ModuleClassLoader for Module "org.hibernate:main" from local module loader 3. sun.misc.Launcher$AppClassLoader Number 1 has all other KeyCloak modules in it, so the entity classes from model-jpa will be found, but the wildfly-extensions module is missing, so entities in classes in a jar in the providers folder cannot be found. Now you guys obviously know a lot more about these internals, but as currently configured, it seems to me there is no way to let Hibernate 'see' these extra classes, since only the KeyCloak services module has a dependency on wildfly-extensions. So I think it boils down to these decisions: A. Do you accept a non-pure-JPA way of building the EntityManagerFactory that has some ties to the Hibernate internals? B. If A is no, than we're done. If yes, then you must find some way to get the extra configured classes 'into' Hibernate. You could get the wildfly-extensions module into scope of the Hibernate classloading. There are serveral ways to configure Hibernate classloading or you could flip some switches / dependencies in the module configuration. Another alternative is to create a separate 'dropfolder' besides themes and providers for JPA extensions, like 'models' or so and have that one be on the Hibernate classpath. But I don't know the exact design principles behind KeyCloak or the Wildfly module system. So maybe you have a better solution or maybe you conclude that this is 'not done' in terms of the architecture. Either way, I'd really appreciate some feedback on this and some thoughts on whether this could be a possible addition to KeyCloak in your eyes. Thanks, Erik Current JPA way. No way to 'interfere': emf = Persistence.createEntityManagerFactory(unitName, properties); Alternative Hibernate only way with adding extra entity class names: // Let Hibernate find and parse all 'persistence.xml' files found on the classpath. List persistenceUnits = PersistenceXmlParser.locatePersistenceUnits(properties); // Assume there is only one persistence unit found and that is the one we need. This can be made more robust by checking on the persistence unit name. ParsedPersistenceXmlDescriptor persistenceUnitDescriptor = persistenceUnits.get(0); // Add extra class names. These could come from a 'JPA class name provider' SPI or something alike. persistenceUnitDescriptor.addClasses("org.keycloak.models.jpa.entities.UserMerchantEntity", "org.keycloak.models.jpa.entities.MerchantEntity"); // Let Hibernate create an EntityManagerFactory out of the (enriched) persistence unit configuration. emf = Bootstrap.getEntityManagerFactoryBuilder(persistenceUnitDescriptor, properties).build(); On 6 November 2015 at 14:29, Erik Mulder <erik.mulder at docdatapayments.com> wrote: Thanks for pointing me explicitly to the SPI documentation. Of course that is exactly what I was looking for in my original question. I don't know how I overlooked this earlier! Probably I was not picking it up, because of almost a decade of developing on Spring projects, where this type of thing works differently. :-) I tried a quick test with a jar with one extra ProtocolMapper configured, put it in the providers folder and it worked like a charm! As for the JPA: We'll probably go with your suggestion of the separate EntityManagerFactory. Indeed there seems to be no way to 'programmatically extend' the list of entity classes in JPA besides editing or overwriting the persistence.xml. As you probably know it can be done in Hibernate, but I guess KeyCloak wants to stick to a generic JPA solution. That said, we might consider the Hibernate specific solution for our case, since being able to switch the JPA provider is not a requirement for us. And keeping the same connection/transaction is a lot easier in reasoning and debugging. We could use Hibernate directly to boostrap as long as it can return an EntityManager. Do you know if that's possible? On 05/11/15 10:52, Stian Thorgersen wrote: The way to extend Keycloak is by implementing your own custom providers of the many SPIs we provide. Some SPIs are more stable (so marked as public) and others are not (so marked as private). If there are things that you want to customize that can't be done with an existing SPI then let us know and we may consider adding additional SPIs. On 4 November 2015 at 17:16, Erik Mulder <erik.mulder at docdatapayments.com> wrote: Thanks for your response! Indeed we already did a proof of concept where we added a custom mapper the way you described (didn't know it was 'protected' territory :). The question is: do we have to override the file 'org.keycloak.protocol.ProtocolMapper' for this and add the new mapper in the original project or is there another way where we don't need to touch the original sources and keep all our changes in a separate project? And how can we do it such that it stays easy to upgrade to newer KeyCloak releases? Each jar has it's own org.keycloak.protocol.ProtocolMapper. Take a look at the docs (http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html) and examples for other provider (https://github.com/keycloak/keycloak/blob/master/examples/providers/event-listener-sysout/src/main/resources/META-INF/services/org.keycloak.events.EventListenerProviderFactory) As for JPA: it would be easier to integrate with the existing JPA project. Again we are wondering whether to start modifying original sources (like persistence.xml) or try to 'externalize' our changes somehow and integrate them using existing 'hooks' in the system or maybe merge projects during build. Maybe there is no good answer to this and we'll always be having some manual merge pains when upgrading to new KeyCloak versions. We just wanted to check if there are preferred ways to add functionality with the least amount of impact on the original sources. I initially wanted the ability to add custom entities to the JpaConnectProvider, but couldn't find a way to define entities programatically with JPA. To add your own persistence.xml you would have to define your own implementation of JpaConnectionProvider and change what is loaded by default (connectionsJpa/provider attribute in keycloak-server.json). Alternative, which is cleaner, but you end up with separate connection/transaction, is to create your own EntityManagerFactory. If it's only used by one provider (for example a custom UserFederationProvider) there's no need to add a connect provider (that's just a way to share one EntityManagerFactory between multiple providers) and you can just create it in the MyUserFederationProviderFactory. On 04/11/15 15:30, Bill Burke wrote: > Custom mappers should be possible. I didn't document it as I wasn't > sure if we wanted to make the SPI public. Custom mappers should just > follow the Provider SPI and they will be picked up. If you see the > META-INF/services/... file in the resources directory of the "services" > or "broker" modules you'll see how to set this up. > > As for extending the JPA datamodel, what you could do is write a new JPA > Connections Provider and plug that in. See connections/jpa. I'm not > sure how you would handle the liquibase db migration. > > On 11/4/2015 6:03 AM, Erik Mulder wrote: >> Hi everybody, >> >> Quick intro: I?m part of a development team in The Netherlands that is >> building a company-wide SSO solution. We?ve chosen KeyCloak to realize >> this and will use OpenID Connect to secure our REST services. It?s a >> great product and seems to be the only one having both support for all >> kinds of security standards and a model and GUI for users and roles. >> Thanks for creating it! J >> >> (if this should be asked instead on the users mailing list, please >> correct me and I?ll post it there) >> >> So far, so good, but we have some extra requirements that do not fit >> into the base KeyCloak data model. See below for details if you?re >> interested. My question is: what is the preferred way / best practice to >> extend the functionality of KeyCloak while keeping the impact on the >> original sources to a minimum? Of course we could just fork the most >> recent version and start hacking away, but we?d like to be able to >> upgrade to newer versions of KeyCloak without too much hassle. >> Possibilities that we?ve come up with so far: >> >> 1.Create completely separate modules that will extend the functionality >> the way we need. >> >> 2.Fork on Github, apply custom changes, and try to merge in updates from >> the master / release branches / tags >> >> 3.Apply custom changes on KeyCloak artifacts using a Maven plugin, such >> as Truezip >> (http://www.mojohaus.org/truezip/truezip-maven-plugin/index.html) - >> manipulate zip files by adding/removing/replacing or Shade >> (http://maven.apache.org/plugins/maven-shade-plugin/) - combine multiple >> jars to 1 'uber-jar' containing the contents of both and when >> overlapping decide on conflicts through configuration. >> >> Of course number 1 is preferred, but I do not see how to add custom >> mappers or JPA entities without making changes in the original module >> files. The other options seem like valid alternatives, but maybe there >> is better / standard way to do this. So any help / insight / shared >> experience on this is much appreciated, thanks! >> >> Kind regards, >> >> Erik Mulder >> >> Senior Software Engineer >> >> Docdata Payments ? NL >> >> P.S. Details on why we want to extend the KeyCloak data model: (any >> feedback on the contents of this P.S. is also welcome!) >> >> Our clients are merchants that have several webshops. We manage their >> online payments (shopping cart checkout). We want to be able to let a >> merchant manage their own users and let a user have different roles for >> different webshops within the same merchant. The overall possible roles >> are fixed though, no specific roles per merchant. We could create a >> separate realm for every merchant, but then we need to duplicate all >> roles every time. Furthermore, in KeyCloak there is no concept of a role >> within a certain context. This is very understandable, since every >> situation has it?s own requirements. We did a proof of concept by adding >> tables and entities for Merchant, UserMerchant, UserMerchantRole etc. >> and adding a custom mapper that can put this information on the Access >> token. Worked like a charm! But it does need some changes in the >> KeyCloak modules and sources to work, hence the question above. >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151116/beaa4daa/attachment-0001.html From bburke at redhat.com Mon Nov 16 16:00:52 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 16 Nov 2015 16:00:52 -0500 Subject: [keycloak-dev] javadocs missing from website Message-ID: <564A4404.9070607@redhat.com> -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Tue Nov 17 02:28:20 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 17 Nov 2015 08:28:20 +0100 Subject: [keycloak-dev] javadocs missing from website In-Reply-To: <564A4404.9070607@redhat.com> References: <564A4404.9070607@redhat.com> Message-ID: Yep, I know. There was an issue with the javadocs and it wasn't included with the release at all. Will be fixed in 1.7 On 16 November 2015 at 22:00, Bill Burke wrote: > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151117/c42e845b/attachment.html From kunal at plivo.com Tue Nov 17 10:06:40 2015 From: kunal at plivo.com (Kunal K) Date: Tue, 17 Nov 2015 20:36:40 +0530 Subject: [keycloak-dev] KEYCLOAK-1900 - Pluggable password hashing algorithm Message-ID: Hi all, I would like to start a discussion on how to implement - https://issues.jboss.org/browse/KEYCLOAK-1900 I have a django web app and all of my users are in a postgres database with salted passwords hashed using SHA. I have been reading how I can use UserFederation to implement by own credential validation, but the drawback here would be that I'll have to keep maintaining my old database. For starters, I was thinking of replacing all occurrences of Pbkdf2PasswordEncoder with an equivalent SHAPasswordEncoder, which is a very crude approach and I'm not sure if it will even work. After some bit of reading I saw this ticket - https://issues.jboss.org/browse/KEYCLOAK-1900 I would like to implement a custom hashing SPI and would love to get some pointers on how to go about it. Thanks -- *KUNAL KERKAR *| PRODUCT ENGINEER Plivo, Inc. 340 Pine St, San Francisco - 94104, USA Web: www.plivo.com | Twitter: @plivo , @tsudot Free Incoming SMS for All US Short Codes ? Get One Today!? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151117/340f53cc/attachment.html From sthorger at redhat.com Tue Nov 17 10:20:16 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 17 Nov 2015 16:20:16 +0100 Subject: [keycloak-dev] KEYCLOAK-1900 - Pluggable password hashing algorithm In-Reply-To: References: Message-ID: Hi, That would be awesome. First step would be to read http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html to understand how Keycloak provides SPIs. Next thing would be to add: * class PasswordHashSPI * interface PasswordHashProviderFactory * interface PasswordHashProvider These should be added to services module. You would also need to change Pbkdf2PasswordEncoder to be the default implementation. Instead of using Pbkdf2PasswordEncoder directly code should use session.getProvider(PasswordHashProvider.class, algorithm). algorithm should be set to on credential entities (UserCredentialValueModel.algorithm). We also need a mechanism to specify the default algorithm (that would be used when users sets new password and also for existing users in the db). On 17 November 2015 at 16:06, Kunal K wrote: > Hi all, > > I would like to start a discussion on how to implement - > https://issues.jboss.org/browse/KEYCLOAK-1900 > > I have a django web app and all of my users are in a postgres database > with salted passwords hashed using SHA. I have been reading how I can use > UserFederation to implement by own credential validation, but the drawback > here would be that I'll have to keep maintaining my old database. > > For starters, I was thinking of replacing all occurrences of > Pbkdf2PasswordEncoder with an equivalent SHAPasswordEncoder, which is a > very crude approach and I'm not sure if it will even work. After some bit > of reading I saw this ticket - > https://issues.jboss.org/browse/KEYCLOAK-1900 > > I would like to implement a custom hashing SPI and would love to get some > pointers on how to go about it. > > Thanks > > -- > *KUNAL KERKAR *| PRODUCT ENGINEER > Plivo, Inc. 340 Pine St, San Francisco - 94104, USA > Web: www.plivo.com | Twitter: @plivo , @tsudot > > > Free Incoming SMS for All US Short Codes ? Get One Today!? > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151117/9587d71a/attachment.html From bruno at abstractj.org Tue Nov 17 10:33:38 2015 From: bruno at abstractj.org (Bruno Oliveira) Date: Tue, 17 Nov 2015 15:33:38 +0000 Subject: [keycloak-dev] KEYCLOAK-1900 - Pluggable password hashing algorithm In-Reply-To: References: Message-ID: By salted passwords using SHA1, do you mean something like: hash(salt + password) ? If yes, hashes like SHA for example, were designed to be fast and can be broken with much less computational power than BCrypt, PBKDF2 or Scrypt for example. On Tue, Nov 17, 2015 at 1:07 PM Kunal K wrote: > Hi all, > > I would like to start a discussion on how to implement - > https://issues.jboss.org/browse/KEYCLOAK-1900 > > I have a django web app and all of my users are in a postgres database > with salted passwords hashed using SHA. I have been reading how I can use > UserFederation to implement by own credential validation, but the drawback > here would be that I'll have to keep maintaining my old database. > > For starters, I was thinking of replacing all occurrences of > Pbkdf2PasswordEncoder with an equivalent SHAPasswordEncoder, which is a > very crude approach and I'm not sure if it will even work. After some bit > of reading I saw this ticket - > https://issues.jboss.org/browse/KEYCLOAK-1900 > > I would like to implement a custom hashing SPI and would love to get some > pointers on how to go about it. > > Thanks > > -- > *KUNAL KERKAR *| PRODUCT ENGINEER > Plivo, Inc. 340 Pine St, San Francisco - 94104, USA > Web: www.plivo.com | Twitter: @plivo , @tsudot > > > Free Incoming SMS for All US Short Codes ? Get One Today!? > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151117/a7b46253/attachment-0001.html From bburke at redhat.com Tue Nov 17 10:41:55 2015 From: bburke at redhat.com (Bill Burke) Date: Tue, 17 Nov 2015 10:41:55 -0500 Subject: [keycloak-dev] KEYCLOAK-1900 - Pluggable password hashing algorithm In-Reply-To: References: Message-ID: <564B4AC3.5080702@redhat.com> can just handle something like this like we do hash iterations. Store the algorithm used, verify the password, compute the new hash with the new algorithm. On 11/17/2015 10:33 AM, Bruno Oliveira wrote: > By salted passwords using SHA1, do you mean something like: > > hash(salt + password) ? > > If yes, hashes like SHA for example, were designed to be fast and can be > broken with much less computational power than BCrypt, PBKDF2 or Scrypt > for example. > > > On Tue, Nov 17, 2015 at 1:07 PM Kunal K > wrote: > > Hi all, > > I would like to start a discussion on how to implement - > https://issues.jboss.org/browse/KEYCLOAK-1900 > > I have a django web app and all of my users are in a postgres > database with salted passwords hashed using SHA. I have been reading > how I can use UserFederation to implement by own credential > validation, but the drawback here would be that I'll have to keep > maintaining my old database. > > For starters, I was thinking of replacing all occurrences of > Pbkdf2PasswordEncoder with an equivalent SHAPasswordEncoder, which > is a very crude approach and I'm not sure if it will even work. > After some bit of reading I saw this ticket - > https://issues.jboss.org/browse/KEYCLOAK-1900 > > I would like to implement a custom hashing SPI and would love to get > some pointers on how to go about it. > > Thanks > > -- > *KUNAL KERKAR *| PRODUCT ENGINEER > Plivo, Inc. 340 Pine St, San Francisco - 94104, USA > Web: www.plivo.com | Twitter: @plivo > , @tsudot > > Free Incoming SMS for All US Short Codes ? Get One Today!? > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bruno at abstractj.org Tue Nov 17 10:48:36 2015 From: bruno at abstractj.org (Bruno Oliveira) Date: Tue, 17 Nov 2015 15:48:36 +0000 Subject: [keycloak-dev] KEYCLOAK-1900 - Pluggable password hashing algorithm In-Reply-To: References: Message-ID: What you mean is migrate from badly broken legacies like: MD5(salt + password) SHA1(salt +password) To BCrypt, Scrypt or PBKDF2? If yes, +1000000 On Tue, Nov 17, 2015 at 1:07 PM Kunal K wrote: > Hi all, > > I would like to start a discussion on how to implement - > https://issues.jboss.org/browse/KEYCLOAK-1900 > > I have a django web app and all of my users are in a postgres database > with salted passwords hashed using SHA. I have been reading how I can use > UserFederation to implement by own credential validation, but the drawback > here would be that I'll have to keep maintaining my old database. > > For starters, I was thinking of replacing all occurrences of > Pbkdf2PasswordEncoder with an equivalent SHAPasswordEncoder, which is a > very crude approach and I'm not sure if it will even work. After some bit > of reading I saw this ticket - > https://issues.jboss.org/browse/KEYCLOAK-1900 > > I would like to implement a custom hashing SPI and would love to get some > pointers on how to go about it. > > Thanks > > -- > *KUNAL KERKAR *| PRODUCT ENGINEER > Plivo, Inc. 340 Pine St, San Francisco - 94104, USA > Web: www.plivo.com | Twitter: @plivo , @tsudot > > > Free Incoming SMS for All US Short Codes ? Get One Today!? > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151117/75494354/attachment.html From kunal at plivo.com Tue Nov 17 12:36:56 2015 From: kunal at plivo.com (Kunal K) Date: Tue, 17 Nov 2015 23:06:56 +0530 Subject: [keycloak-dev] KEYCLOAK-1900 - Pluggable password hashing algorithm In-Reply-To: References: Message-ID: Oh yes SHA1 can certainly be compromised. I feel a better approach to migrate existing passwords would be like: BCrypt(SHA1(salt+password)) On Tue, Nov 17, 2015 at 9:18 PM, Bruno Oliveira wrote: > What you mean is migrate from badly broken legacies like: > > MD5(salt + password) > SHA1(salt +password) > > To BCrypt, Scrypt or PBKDF2? If yes, +1000000 > > On Tue, Nov 17, 2015 at 1:07 PM Kunal K wrote: > >> Hi all, >> >> I would like to start a discussion on how to implement - >> https://issues.jboss.org/browse/KEYCLOAK-1900 >> >> I have a django web app and all of my users are in a postgres database >> with salted passwords hashed using SHA. I have been reading how I can use >> UserFederation to implement by own credential validation, but the drawback >> here would be that I'll have to keep maintaining my old database. >> >> For starters, I was thinking of replacing all occurrences of >> Pbkdf2PasswordEncoder with an equivalent SHAPasswordEncoder, which is a >> very crude approach and I'm not sure if it will even work. After some bit >> of reading I saw this ticket - >> https://issues.jboss.org/browse/KEYCLOAK-1900 >> >> I would like to implement a custom hashing SPI and would love to get some >> pointers on how to go about it. >> >> Thanks >> >> -- >> *KUNAL KERKAR *| PRODUCT ENGINEER >> Plivo, Inc. 340 Pine St, San Francisco - 94104, USA >> Web: www.plivo.com | Twitter: @plivo , @tsudot >> >> >> Free Incoming SMS for All US Short Codes ? Get One Today!? >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -- *KUNAL KERKAR *| PRODUCT ENGINEER Plivo, Inc. 340 Pine St, San Francisco - 94104, USA Web: www.plivo.com | Twitter: @plivo , @tsudot Free Incoming SMS for All US Short Codes ? Get One Today!? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151117/0b4ac643/attachment.html From kunal at plivo.com Tue Nov 17 12:37:56 2015 From: kunal at plivo.com (Kunal K) Date: Tue, 17 Nov 2015 23:07:56 +0530 Subject: [keycloak-dev] KEYCLOAK-1900 - Pluggable password hashing algorithm In-Reply-To: References: Message-ID: Thanks for those notes Stian, I will read up and document my progress on this thread. On Tue, Nov 17, 2015 at 8:50 PM, Stian Thorgersen wrote: > Hi, > > That would be awesome. > > First step would be to read > http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html > to understand how Keycloak provides SPIs. > > Next thing would be to add: > > * class PasswordHashSPI > * interface PasswordHashProviderFactory > * interface PasswordHashProvider > > These should be added to services module. You would also need to > change Pbkdf2PasswordEncoder to be the default implementation. > > Instead of using Pbkdf2PasswordEncoder directly code should use > session.getProvider(PasswordHashProvider.class, algorithm). algorithm > should be set to on credential entities > (UserCredentialValueModel.algorithm). We also need a mechanism to specify > the default algorithm (that would be used when users sets new password and > also for existing users in the db). > > > On 17 November 2015 at 16:06, Kunal K wrote: > >> Hi all, >> >> I would like to start a discussion on how to implement - >> https://issues.jboss.org/browse/KEYCLOAK-1900 >> >> I have a django web app and all of my users are in a postgres database >> with salted passwords hashed using SHA. I have been reading how I can use >> UserFederation to implement by own credential validation, but the drawback >> here would be that I'll have to keep maintaining my old database. >> >> For starters, I was thinking of replacing all occurrences of >> Pbkdf2PasswordEncoder with an equivalent SHAPasswordEncoder, which is a >> very crude approach and I'm not sure if it will even work. After some bit >> of reading I saw this ticket - >> https://issues.jboss.org/browse/KEYCLOAK-1900 >> >> I would like to implement a custom hashing SPI and would love to get some >> pointers on how to go about it. >> >> Thanks >> >> -- >> *KUNAL KERKAR *| PRODUCT ENGINEER >> Plivo, Inc. 340 Pine St, San Francisco - 94104, USA >> Web: www.plivo.com | Twitter: @plivo , @tsudot >> >> >> Free Incoming SMS for All US Short Codes ? Get One Today!? >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > -- *KUNAL KERKAR *| PRODUCT ENGINEER Plivo, Inc. 340 Pine St, San Francisco - 94104, USA Web: www.plivo.com | Twitter: @plivo , @tsudot Free Incoming SMS for All US Short Codes ? Get One Today!? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151117/ed79b7c7/attachment-0001.html From sthorger at redhat.com Wed Nov 18 03:33:46 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 18 Nov 2015 09:33:46 +0100 Subject: [keycloak-dev] Admin console doesn't reload messages at runtime Message-ID: The admin console doesn't currently reload messages at runtime. This just doesn't work for development as you have to reload the server each time to get new messages. Any caching of messages (or theme resources in general) should be added to ExtendingThemeManager and made available to all uses of a theme. ExtendingThemeManager already has some caching capability and it's configurable so can be disabled during development. Caching of message bundles would actually be much more important to have for login screens than for the admin console. For now I'm removing AdminMessagesLoader and I'll add caching to ExtendingThemeManager soon. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151118/30787411/attachment.html From sthorger at redhat.com Wed Nov 18 04:25:39 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 18 Nov 2015 10:25:39 +0100 Subject: [keycloak-dev] Angular Translate from controllers and services Message-ID: There are no examples on how to use translations from within services and controllers. $translate('label') doesn't work as it just returns a promise. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151118/d1f25647/attachment.html From gerbermichi at me.com Wed Nov 18 04:58:51 2015 From: gerbermichi at me.com (Michael Gerber) Date: Wed, 18 Nov 2015 09:58:51 +0000 (GMT) Subject: [keycloak-dev] =?utf-8?q?_Re=3A__Angular_Translate_from_controlle?= =?utf-8?q?rs_and_services?= Message-ID: <4d7b1e1d-e74f-4649-9042-f2676634edbc@me.com> $translate.instant('label')? You can find the doc here: https://angular-translate.github.io/docs/#/api/pascalprecht.translate.$translate Am 18. November 2015 um 10:26 schrieb Stian Thorgersen : There are no examples on how to use translations from within services and controllers. $translate('label') doesn't work as it just returns a promise. _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151118/9f5badbd/attachment.html From sthorger at redhat.com Wed Nov 18 07:24:34 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 18 Nov 2015 13:24:34 +0100 Subject: [keycloak-dev] Angular Translate from controllers and services In-Reply-To: <4d7b1e1d-e74f-4649-9042-f2676634edbc@me.com> References: <4d7b1e1d-e74f-4649-9042-f2676634edbc@me.com> Message-ID: Works like a charm - thanks :) On 18 November 2015 at 10:58, Michael Gerber wrote: > $translate.instant('label') > > You can find the doc here: > > https://angular-translate.github.io/docs/#/api/pascalprecht.translate.$translate > > Am 18. November 2015 um 10:26 schrieb Stian Thorgersen < > sthorger at redhat.com>: > > There are no examples on how to use translations from within services and > controllers. > > $translate('label') doesn't work as it just returns a promise. > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151118/3f5c5fa5/attachment.html From ssilvert at redhat.com Wed Nov 18 07:39:12 2015 From: ssilvert at redhat.com (Stan Silvert) Date: Wed, 18 Nov 2015 07:39:12 -0500 Subject: [keycloak-dev] Angular Translate from controllers and services In-Reply-To: References: Message-ID: <564C7170.90807@redhat.com> Does this help? https://angular-translate.github.io/docs/#/guide/03_using-translate-service On 11/18/2015 4:25 AM, Stian Thorgersen wrote: > There are no examples on how to use translations from within services > and controllers. > > $translate('label') doesn't work as it just returns a promise. > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151118/2d096552/attachment.html From kunal at plivo.com Wed Nov 18 09:09:44 2015 From: kunal at plivo.com (Kunal K) Date: Wed, 18 Nov 2015 19:39:44 +0530 Subject: [keycloak-dev] KEYCLOAK-1900 - Pluggable password hashing algorithm In-Reply-To: References: Message-ID: Hi Stian, Could you please review this code - https://github.com/tsudot/keycloak/commit/ce58d795bfea9e6c19663fa40d7a499d2d78aeab I'm having trouble figuring out how to call session.getProvider(PasswordHashProvider.class, algorithm) to replace Pbkdf2PasswordEncoder. I checked https://github.com/tsudot/keycloak/blob/master/model/jpa/src/main/java/org/keycloak/models/jpa/UserAdapter.java#L399 but couldn't find any instance of KeycloakSession. Am I missing something? On Tue, Nov 17, 2015 at 11:07 PM, Kunal K wrote: > Thanks for those notes Stian, I will read up and document my progress on > this thread. > > On Tue, Nov 17, 2015 at 8:50 PM, Stian Thorgersen > wrote: > >> Hi, >> >> That would be awesome. >> >> First step would be to read >> http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html >> to understand how Keycloak provides SPIs. >> >> Next thing would be to add: >> >> * class PasswordHashSPI >> * interface PasswordHashProviderFactory >> * interface PasswordHashProvider >> >> These should be added to services module. You would also need to >> change Pbkdf2PasswordEncoder to be the default implementation. >> >> Instead of using Pbkdf2PasswordEncoder directly code should use >> session.getProvider(PasswordHashProvider.class, algorithm). algorithm >> should be set to on credential entities >> (UserCredentialValueModel.algorithm). We also need a mechanism to specify >> the default algorithm (that would be used when users sets new password and >> also for existing users in the db). >> >> >> On 17 November 2015 at 16:06, Kunal K wrote: >> >>> Hi all, >>> >>> I would like to start a discussion on how to implement - >>> https://issues.jboss.org/browse/KEYCLOAK-1900 >>> >>> I have a django web app and all of my users are in a postgres database >>> with salted passwords hashed using SHA. I have been reading how I can use >>> UserFederation to implement by own credential validation, but the drawback >>> here would be that I'll have to keep maintaining my old database. >>> >>> For starters, I was thinking of replacing all occurrences of >>> Pbkdf2PasswordEncoder with an equivalent SHAPasswordEncoder, which is a >>> very crude approach and I'm not sure if it will even work. After some bit >>> of reading I saw this ticket - >>> https://issues.jboss.org/browse/KEYCLOAK-1900 >>> >>> I would like to implement a custom hashing SPI and would love to get >>> some pointers on how to go about it. >>> >>> Thanks >>> >>> -- >>> *KUNAL KERKAR *| PRODUCT ENGINEER >>> Plivo, Inc. 340 Pine St, San Francisco - 94104, USA >>> Web: www.plivo.com | Twitter: @plivo , @tsudot >>> >>> >>> Free Incoming SMS for All US Short Codes ? Get One Today!? >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> > > > -- > *KUNAL KERKAR *| PRODUCT ENGINEER > Plivo, Inc. 340 Pine St, San Francisco - 94104, USA > Web: www.plivo.com | Twitter: @plivo , @tsudot > > > Free Incoming SMS for All US Short Codes ? Get One Today!? > > -- *KUNAL KERKAR *| PRODUCT ENGINEER Plivo, Inc. 340 Pine St, San Francisco - 94104, USA Web: www.plivo.com | Twitter: @plivo , @tsudot Free Incoming SMS for All US Short Codes ? Get One Today!? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151118/56f7e52c/attachment-0001.html From gerbermichi at me.com Wed Nov 18 11:22:09 2015 From: gerbermichi at me.com (Michael Gerber) Date: Wed, 18 Nov 2015 16:22:09 +0000 (GMT) Subject: [keycloak-dev] restart the authentication flow after a timeout error Message-ID: Hi all, I am using a client with urn:ietf:wg:oauth:2.0:oob. If a user has?too long to change his password, he will retrieve an error message.?However, he can not restart the authentication process.? As far as I understand the verifyAction method in the LoginActionsService, only the "authentication" action will reset the authentication flow. I think it would make sense to reset the authentication flow also if a user has too long to change his password or so.? What do you think? Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151118/8f3d3839/attachment.html From bburke at redhat.com Wed Nov 18 11:28:25 2015 From: bburke at redhat.com (Bill Burke) Date: Wed, 18 Nov 2015 11:28:25 -0500 Subject: [keycloak-dev] mongo from Intellij? In-Reply-To: <564CA50D.7080409@redhat.com> References: <564CA50D.7080409@redhat.com> Message-ID: <564CA729.9080401@redhat.com> Anybody know how to run mongo backend with integration tests within Intellij? -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Wed Nov 18 15:11:23 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 18 Nov 2015 21:11:23 +0100 Subject: [keycloak-dev] mongo from Intellij? In-Reply-To: <564CA729.9080401@redhat.com> References: <564CA50D.7080409@redhat.com> <564CA729.9080401@redhat.com> Message-ID: You need Mongo running on your machine first as the embedded Mongo used for the testsuite is ran from Maven. Once you have Mongo running locally it's pretty easy, edit (but don't commit the changes): testsuite/integration/src/test/resources/META-INF/keycloak-server.json Change the providers to mongo. There are default settings for Mongo already there and should work as long as you have Mongo running locally. I find it easier to just change the keycloak-server.json directly, but you can also use system properties to set the providers, but that's pretty annoying to do when your running tests from IntelliJ. On 18 November 2015 at 17:28, Bill Burke wrote: > Anybody know how to run mongo backend with integration tests within > Intellij? > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151118/12903d25/attachment.html From sthorger at redhat.com Thu Nov 19 03:09:19 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 19 Nov 2015 09:09:19 +0100 Subject: [keycloak-dev] Keycloak 1.7 CR1 release update Message-ID: Keycloak 1.7 CR1 is scheduled to be released on 2nd December (Wed). I'd like everything to be done by end of 30th November (Mon). Then we'll do some testing on Tuesday. Unless any major issues are reported Keycloak 1.7 Final will be released the following week. For releases in the future we'll aim for the following: * Complete work by end of Monday * Test on Tuesday * Release CR1 on Wednesday (Friday at the latest) * New CR release ASAP as major issues are fixed * Release Final one week after last CR -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/f1d9200b/attachment.html From sthorger at redhat.com Thu Nov 19 05:34:03 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 19 Nov 2015 11:34:03 +0100 Subject: [keycloak-dev] KEYCLOAK-1900 - Pluggable password hashing algorithm In-Reply-To: References: Message-ID: Looks good. You also need to add META-INF/services/org.keycloak.service.PasswordHashProviderFactory and add org.keycloak.service.DefaultPasswordHashProviderFactory. Please also move the classes to a separate package, 'org.keycloak.hash' would work. You need to add the KeycloakSession to the UserAdapter constructor. On 18 November 2015 at 15:09, Kunal K wrote: > Hi Stian, > > Could you please review this code - > https://github.com/tsudot/keycloak/commit/ce58d795bfea9e6c19663fa40d7a499d2d78aeab > > I'm having trouble figuring out how to call session.getProvider(PasswordHashProvider.class, > algorithm) to replace Pbkdf2PasswordEncoder. > > I checked > https://github.com/tsudot/keycloak/blob/master/model/jpa/src/main/java/org/keycloak/models/jpa/UserAdapter.java#L399 > but couldn't find any instance of KeycloakSession. Am I missing something? > > On Tue, Nov 17, 2015 at 11:07 PM, Kunal K wrote: > >> Thanks for those notes Stian, I will read up and document my progress on >> this thread. >> >> On Tue, Nov 17, 2015 at 8:50 PM, Stian Thorgersen >> wrote: >> >>> Hi, >>> >>> That would be awesome. >>> >>> First step would be to read >>> http://keycloak.github.io/docs/userguide/keycloak-server/html/providers.html >>> to understand how Keycloak provides SPIs. >>> >>> Next thing would be to add: >>> >>> * class PasswordHashSPI >>> * interface PasswordHashProviderFactory >>> * interface PasswordHashProvider >>> >>> These should be added to services module. You would also need to >>> change Pbkdf2PasswordEncoder to be the default implementation. >>> >>> Instead of using Pbkdf2PasswordEncoder directly code should use >>> session.getProvider(PasswordHashProvider.class, algorithm). algorithm >>> should be set to on credential entities >>> (UserCredentialValueModel.algorithm). We also need a mechanism to specify >>> the default algorithm (that would be used when users sets new password and >>> also for existing users in the db). >>> >>> >>> On 17 November 2015 at 16:06, Kunal K wrote: >>> >>>> Hi all, >>>> >>>> I would like to start a discussion on how to implement - >>>> https://issues.jboss.org/browse/KEYCLOAK-1900 >>>> >>>> I have a django web app and all of my users are in a postgres database >>>> with salted passwords hashed using SHA. I have been reading how I can use >>>> UserFederation to implement by own credential validation, but the drawback >>>> here would be that I'll have to keep maintaining my old database. >>>> >>>> For starters, I was thinking of replacing all occurrences of >>>> Pbkdf2PasswordEncoder with an equivalent SHAPasswordEncoder, which is a >>>> very crude approach and I'm not sure if it will even work. After some bit >>>> of reading I saw this ticket - >>>> https://issues.jboss.org/browse/KEYCLOAK-1900 >>>> >>>> I would like to implement a custom hashing SPI and would love to get >>>> some pointers on how to go about it. >>>> >>>> Thanks >>>> >>>> -- >>>> *KUNAL KERKAR *| PRODUCT ENGINEER >>>> Plivo, Inc. 340 Pine St, San Francisco - 94104, USA >>>> Web: www.plivo.com | Twitter: @plivo , >>>> @tsudot >>>> >>>> Free Incoming SMS for All US Short Codes ? Get One Today!? >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>> >>> >> >> >> -- >> *KUNAL KERKAR *| PRODUCT ENGINEER >> Plivo, Inc. 340 Pine St, San Francisco - 94104, USA >> Web: www.plivo.com | Twitter: @plivo , @tsudot >> >> >> Free Incoming SMS for All US Short Codes ? Get One Today!? >> >> > > > > -- > *KUNAL KERKAR *| PRODUCT ENGINEER > Plivo, Inc. 340 Pine St, San Francisco - 94104, USA > Web: www.plivo.com | Twitter: @plivo , @tsudot > > > Free Incoming SMS for All US Short Codes ? Get One Today!? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/922409a7/attachment-0001.html From mposolda at redhat.com Thu Nov 19 07:52:59 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 19 Nov 2015 13:52:59 +0100 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: References: Message-ID: <564DC62B.7020205@redhat.com> +1 for this change. I am just not sure if we should set the "id" to the current value of "client-id" ? Few things to note: - SAML clients currently use clientId in form of URL. For example in SAML demo there are clientIds like "http://localhosT:8080/employee-sig" . I don't know if it's requirement, maybe it's possible to solve it somehow (ie. introduce different attribute for SAML client to store these URLs). But from what I remember, Bill changed admin console to use "id" instead of "clientId" because there were issues with URL-like clientId in admin console . So if we overwrite the "id" with current "client-id" the issue will be back. - Migration might be a pain. Many tables (roles, protocolMappers, user consents, offline clientSessions ...) references client by "id" . Overwriting "id" with "client-id" means that we will need to change all those DB records. And there are things like foreign keys etc... Shouldn't do vice-versa and just remove current "client-id" and ask people to update their keycloak.json adapter configurations? On the other hand, removing "client-id" might break migration of JSON exported realms as the JSON entities are using "client-id" for referencing client. It seems the migration will be a pain regardless of whatever direction we choose :-( Marek On 16/11/15 14:54, Stian Thorgersen wrote: > We have both "id" and "client-id" for clients in Keycloak at the > moment. This seems unnecessary and complex. > > The model can retrieve clients on either value. In token endpoints the > "client-id" is used. In admin endpoints the "id" is used. > > Also, in most cases it would be simpler for users to just have a > generated id than having to come up with one themselves. The id > doesn't have to be human readable either as we have name for that. > > OpenID Connect expects "client-id" to be generated by the IdP and > can't be changed once created. > > I propose we remove "client-id" and only keep id. > > For migration of existing clients we would set the "id" value to the > current value of "client-id". This would require no changes to adapter > configs. When creating new clients from the admin console we would not > allow setting the "client-id", instead just display it after the > client was created. When importing clients it would be possible to set > the id (and for backwards compatibility we would set "id" equal to the > "client-id" field. > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/7924b98b/attachment.html From mposolda at redhat.com Thu Nov 19 07:59:59 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 19 Nov 2015 13:59:59 +0100 Subject: [keycloak-dev] mongo from Intellij? In-Reply-To: References: <564CA50D.7080409@redhat.com> <564CA729.9080401@redhat.com> Message-ID: <564DC7CF.70005@redhat.com> On 18/11/15 21:11, Stian Thorgersen wrote: > You need Mongo running on your machine first as the embedded Mongo > used for the testsuite is ran from Maven. > > Once you have Mongo running locally it's pretty easy, edit (but don't > commit the changes): > > testsuite/integration/src/test/resources/META-INF/keycloak-server.json > > Change the providers to mongo. There are default settings for Mongo > already there and should work as long as you have Mongo running locally. > > I find it easier to just change the keycloak-server.json directly, but > you can also use system properties to set the providers, but that's > pretty annoying to do when your running tests from IntelliJ. For me, it doesn't seem annoying. I just copy/paste this line with properties when running any test: -Dkeycloak.realm.provider=mongo -Dkeycloak.user.provider=mongo -Dkeycloak.userSessionPersister.provider=mongo -Dkeycloak.eventsStore.provider=mongo -Dkeycloak.connectionsMongo.db=keycloak -Dresources I also usually use this for KeycloakServer setup during development. Marek > > On 18 November 2015 at 17:28, Bill Burke > wrote: > > Anybody know how to run mongo backend with integration tests within > Intellij? > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/c8bf9567/attachment.html From bburke at redhat.com Thu Nov 19 08:17:41 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 19 Nov 2015 08:17:41 -0500 Subject: [keycloak-dev] mongo from Intellij? In-Reply-To: <564DC7CF.70005@redhat.com> References: <564CA50D.7080409@redhat.com> <564CA729.9080401@redhat.com> <564DC7CF.70005@redhat.com> Message-ID: <564DCBF5.8070505@redhat.com> We have some script to run Mongo? On 11/19/2015 7:59 AM, Marek Posolda wrote: > On 18/11/15 21:11, Stian Thorgersen wrote: >> You need Mongo running on your machine first as the embedded Mongo >> used for the testsuite is ran from Maven. >> >> Once you have Mongo running locally it's pretty easy, edit (but don't >> commit the changes): >> >> testsuite/integration/src/test/resources/META-INF/keycloak-server.json >> >> Change the providers to mongo. There are default settings for Mongo >> already there and should work as long as you have Mongo running locally. >> >> I find it easier to just change the keycloak-server.json directly, but >> you can also use system properties to set the providers, but that's >> pretty annoying to do when your running tests from IntelliJ. > For me, it doesn't seem annoying. I just copy/paste this line with > properties when running any test: > > -Dkeycloak.realm.provider=mongo -Dkeycloak.user.provider=mongo > -Dkeycloak.userSessionPersister.provider=mongo > -Dkeycloak.eventsStore.provider=mongo > -Dkeycloak.connectionsMongo.db=keycloak -Dresources > > I also usually use this for KeycloakServer setup during development. > > Marek > >> >> On 18 November 2015 at 17:28, Bill Burke > > wrote: >> >> Anybody know how to run mongo backend with integration tests within >> Intellij? >> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Thu Nov 19 08:17:48 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 19 Nov 2015 14:17:48 +0100 Subject: [keycloak-dev] mongo from Intellij? In-Reply-To: <564DC7CF.70005@redhat.com> References: <564CA50D.7080409@redhat.com> <564CA729.9080401@redhat.com> <564DC7CF.70005@redhat.com> Message-ID: Actually you can also add those properties to "/.keycloak-server.properties" as KeycloakServer loads those at runtime. On 19 November 2015 at 13:59, Marek Posolda wrote: > On 18/11/15 21:11, Stian Thorgersen wrote: > > You need Mongo running on your machine first as the embedded Mongo used > for the testsuite is ran from Maven. > > Once you have Mongo running locally it's pretty easy, edit (but don't > commit the changes): > > testsuite/integration/src/test/resources/META-INF/keycloak-server.json > > Change the providers to mongo. There are default settings for Mongo > already there and should work as long as you have Mongo running locally. > > I find it easier to just change the keycloak-server.json directly, but you > can also use system properties to set the providers, but that's pretty > annoying to do when your running tests from IntelliJ. > > For me, it doesn't seem annoying. I just copy/paste this line with > properties when running any test: > > -Dkeycloak.realm.provider=mongo -Dkeycloak.user.provider=mongo > -Dkeycloak.userSessionPersister.provider=mongo > -Dkeycloak.eventsStore.provider=mongo > -Dkeycloak.connectionsMongo.db=keycloak -Dresources > > I also usually use this for KeycloakServer setup during development. > > Marek > > > On 18 November 2015 at 17:28, Bill Burke wrote: > >> Anybody know how to run mongo backend with integration tests within >> Intellij? >> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/6c77fed0/attachment.html From sthorger at redhat.com Thu Nov 19 08:18:19 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 19 Nov 2015 14:18:19 +0100 Subject: [keycloak-dev] mongo from Intellij? In-Reply-To: <564DCBF5.8070505@redhat.com> References: <564CA50D.7080409@redhat.com> <564CA729.9080401@redhat.com> <564DC7CF.70005@redhat.com> <564DCBF5.8070505@redhat.com> Message-ID: No, but I think it should be relatively easy to run the embedded Mongo from Maven On 19 November 2015 at 14:17, Bill Burke wrote: > We have some script to run Mongo? > > On 11/19/2015 7:59 AM, Marek Posolda wrote: > >> On 18/11/15 21:11, Stian Thorgersen wrote: >> >>> You need Mongo running on your machine first as the embedded Mongo >>> used for the testsuite is ran from Maven. >>> >>> Once you have Mongo running locally it's pretty easy, edit (but don't >>> commit the changes): >>> >>> testsuite/integration/src/test/resources/META-INF/keycloak-server.json >>> >>> Change the providers to mongo. There are default settings for Mongo >>> already there and should work as long as you have Mongo running locally. >>> >>> I find it easier to just change the keycloak-server.json directly, but >>> you can also use system properties to set the providers, but that's >>> pretty annoying to do when your running tests from IntelliJ. >>> >> For me, it doesn't seem annoying. I just copy/paste this line with >> properties when running any test: >> >> -Dkeycloak.realm.provider=mongo -Dkeycloak.user.provider=mongo >> -Dkeycloak.userSessionPersister.provider=mongo >> -Dkeycloak.eventsStore.provider=mongo >> -Dkeycloak.connectionsMongo.db=keycloak -Dresources >> >> I also usually use this for KeycloakServer setup during development. >> >> Marek >> >> >>> On 18 November 2015 at 17:28, Bill Burke >> > wrote: >>> >>> Anybody know how to run mongo backend with integration tests within >>> Intellij? >>> >>> >>> -- >>> Bill Burke >>> JBoss, a division of Red Hat >>> http://bill.burkecentral.com >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/a79d212c/attachment-0001.html From mposolda at redhat.com Thu Nov 19 08:39:29 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 19 Nov 2015 14:39:29 +0100 Subject: [keycloak-dev] Change few methods on UserFederationProvider Message-ID: <564DD111.6090405@redhat.com> Currently we have those methods on UserFederationProvider: boolean validCredentials(RealmModel realm, UserModel user, List input); boolean validCredentials(RealmModel realm, UserModel user, UserCredentialModel... input); I propose if we can: 1) Remove the second one as it's not used from anywhere . 2) Change the signature of first one to return "CredentialValidationOutput" instead of boolean. This will allow federationProvider to send some additional state related to authentication instead of just true/false . The main reason is the https://issues.jboss.org/browse/KEYCLOAK-1744 . Basically ActiveDirectory throws exception with different code if password provided by user is incorrect or if the password is correct, but expired. For writable LDAP, it's fine. If password is expired, we can authenticate user, but put requiredAction for UPDATE_PASSWORD on him. However for read-only LDAP, we can't update password from Keycloak. In this case, it will be nice if we can show the message in UI like "Your password has expired. Contact your administrator to change password" . But that's possible if we send some additional state about the reason of failure, so Authenticator can read it and possibly display various messages based on that. IMO will be cool to have solution for https://issues.jboss.org/browse/KEYCLOAK-1744 available in Keycloak out of the box. There are lot of people using ActiveDirectory and asking for this. WDYT? Marek From bburke at redhat.com Thu Nov 19 08:40:42 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 19 Nov 2015 08:40:42 -0500 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: <564DC62B.7020205@redhat.com> References: <564DC62B.7020205@redhat.com> Message-ID: <564DD15A.9080907@redhat.com> We currently actually have 3 client identifers: ID, ClientID, and Name. Also, I think there are a lot of duplicate clientID names between realms. i.e. "admin-console" "broker", etc... A search by clientID is (realm + clientID). Marek is right about why I switched everything to ID in the admin console. For SAML I just didn't want to add yet another client identifier since we already had 3. On 11/19/2015 7:52 AM, Marek Posolda wrote: > +1 for this change. > > I am just not sure if we should set the "id" to the current value of > "client-id" ? Few things to note: > > - SAML clients currently use clientId in form of URL. For example in > SAML demo there are clientIds like "http://localhosT:8080/employee-sig" > . I don't know if it's requirement, maybe it's possible to solve it > somehow (ie. introduce different attribute for SAML client to store > these URLs). But from what I remember, Bill changed admin console to use > "id" instead of "clientId" because there were issues with URL-like > clientId in admin console . So if we overwrite the "id" with current > "client-id" the issue will be back. > > - Migration might be a pain. Many tables (roles, protocolMappers, user > consents, offline clientSessions ...) references client by "id" . > Overwriting "id" with "client-id" means that we will need to change all > those DB records. And there are things like foreign keys etc... > > Shouldn't do vice-versa and just remove current "client-id" and ask > people to update their keycloak.json adapter configurations? On the > other hand, removing "client-id" might break migration of JSON exported > realms as the JSON entities are using "client-id" for referencing client. > > It seems the migration will be a pain regardless of whatever direction > we choose :-( > > Marek > > On 16/11/15 14:54, Stian Thorgersen wrote: >> We have both "id" and "client-id" for clients in Keycloak at the >> moment. This seems unnecessary and complex. >> >> The model can retrieve clients on either value. In token endpoints the >> "client-id" is used. In admin endpoints the "id" is used. >> >> Also, in most cases it would be simpler for users to just have a >> generated id than having to come up with one themselves. The id >> doesn't have to be human readable either as we have name for that. >> >> OpenID Connect expects "client-id" to be generated by the IdP and >> can't be changed once created. >> >> I propose we remove "client-id" and only keep id. >> >> For migration of existing clients we would set the "id" value to the >> current value of "client-id". This would require no changes to adapter >> configs. When creating new clients from the admin console we would not >> allow setting the "client-id", instead just display it after the >> client was created. When importing clients it would be possible to set >> the id (and for backwards compatibility we would set "id" equal to the >> "client-id" field. >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Thu Nov 19 08:42:08 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 19 Nov 2015 08:42:08 -0500 Subject: [keycloak-dev] Change few methods on UserFederationProvider In-Reply-To: <564DD111.6090405@redhat.com> References: <564DD111.6090405@redhat.com> Message-ID: <564DD1B0.1070006@redhat.com> Sounds good. We're just going to have to be breaking backward compatibility to clean up these SPIs. Sucks, but necessary. On 11/19/2015 8:39 AM, Marek Posolda wrote: > Currently we have those methods on UserFederationProvider: > > boolean validCredentials(RealmModel realm, UserModel user, > List input); > boolean validCredentials(RealmModel realm, UserModel user, > UserCredentialModel... input); > > > I propose if we can: > 1) Remove the second one as it's not used from anywhere . > > 2) Change the signature of first one to return > "CredentialValidationOutput" instead of boolean. This will allow > federationProvider to send some additional state related to > authentication instead of just true/false . > > The main reason is the https://issues.jboss.org/browse/KEYCLOAK-1744 . > Basically ActiveDirectory throws exception with different code if > password provided by user is incorrect or if the password is correct, > but expired. For writable LDAP, it's fine. If password is expired, we > can authenticate user, but put requiredAction for UPDATE_PASSWORD on him. > > However for read-only LDAP, we can't update password from Keycloak. In > this case, it will be nice if we can show the message in UI like "Your > password has expired. Contact your administrator to change password" . > But that's possible if we send some additional state about the reason of > failure, so Authenticator can read it and possibly display various > messages based on that. > > IMO will be cool to have solution for > https://issues.jboss.org/browse/KEYCLOAK-1744 available in Keycloak out > of the box. There are lot of people using ActiveDirectory and asking for > this. > > WDYT? > Marek > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Thu Nov 19 08:49:03 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 19 Nov 2015 08:49:03 -0500 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: <564DD15A.9080907@redhat.com> References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> Message-ID: <564DD34F.603@redhat.com> One more thing. Another problem is that if you switch to ID for everything, then demos etc. will no longer run out of the box and you'll have to modify adapter configuration to set the newly generated DB id. Really, any piece of configuration that references things by client ID will no longer work. So, there's really 2 issues: * Using ClientID in admin-console URLs doesn't work well as the client IDs could have '/' characters in them * Using ID only messes up pre-defined, pre-packaged adapter config and you have to edit these files after the keycloak DB is set up. On 11/19/2015 8:40 AM, Bill Burke wrote: > We currently actually have 3 client identifers: ID, ClientID, and Name. > > Also, I think there are a lot of duplicate clientID names between > realms. i.e. "admin-console" "broker", etc... > > A search by clientID is (realm + clientID). > > Marek is right about why I switched everything to ID in the admin > console. For SAML I just didn't want to add yet another client > identifier since we already had 3. > > On 11/19/2015 7:52 AM, Marek Posolda wrote: >> +1 for this change. >> >> I am just not sure if we should set the "id" to the current value of >> "client-id" ? Few things to note: >> >> - SAML clients currently use clientId in form of URL. For example in >> SAML demo there are clientIds like "http://localhosT:8080/employee-sig" >> . I don't know if it's requirement, maybe it's possible to solve it >> somehow (ie. introduce different attribute for SAML client to store >> these URLs). But from what I remember, Bill changed admin console to use >> "id" instead of "clientId" because there were issues with URL-like >> clientId in admin console . So if we overwrite the "id" with current >> "client-id" the issue will be back. >> >> - Migration might be a pain. Many tables (roles, protocolMappers, user >> consents, offline clientSessions ...) references client by "id" . >> Overwriting "id" with "client-id" means that we will need to change all >> those DB records. And there are things like foreign keys etc... >> >> Shouldn't do vice-versa and just remove current "client-id" and ask >> people to update their keycloak.json adapter configurations? On the >> other hand, removing "client-id" might break migration of JSON exported >> realms as the JSON entities are using "client-id" for referencing client. >> >> It seems the migration will be a pain regardless of whatever direction >> we choose :-( >> >> Marek >> >> On 16/11/15 14:54, Stian Thorgersen wrote: >>> We have both "id" and "client-id" for clients in Keycloak at the >>> moment. This seems unnecessary and complex. >>> >>> The model can retrieve clients on either value. In token endpoints the >>> "client-id" is used. In admin endpoints the "id" is used. >>> >>> Also, in most cases it would be simpler for users to just have a >>> generated id than having to come up with one themselves. The id >>> doesn't have to be human readable either as we have name for that. >>> >>> OpenID Connect expects "client-id" to be generated by the IdP and >>> can't be changed once created. >>> >>> I propose we remove "client-id" and only keep id. >>> >>> For migration of existing clients we would set the "id" value to the >>> current value of "client-id". This would require no changes to adapter >>> configs. When creating new clients from the admin console we would not >>> allow setting the "client-id", instead just display it after the >>> client was created. When importing clients it would be possible to set >>> the id (and for backwards compatibility we would set "id" equal to the >>> "client-id" field. >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From mposolda at redhat.com Thu Nov 19 08:49:03 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 19 Nov 2015 14:49:03 +0100 Subject: [keycloak-dev] mongo from Intellij? In-Reply-To: References: <564CA50D.7080409@redhat.com> <564CA729.9080401@redhat.com> <564DC7CF.70005@redhat.com> <564DCBF5.8070505@redhat.com> Message-ID: <564DD34F.8040101@redhat.com> I just have mongo running locally on my laptop and using that for development. Once you have mongo installed, you run command "mongod" and you're done. The maven "mongo" profile is using the embedded Mongo executed on port 27018 to not clash with the default mongod running on 27017 . That allows to run mongo testsuite from maven on laptops, which don't have mongo installed. Marek On 19/11/15 14:18, Stian Thorgersen wrote: > No, but I think it should be relatively easy to run the embedded Mongo > from Maven > > On 19 November 2015 at 14:17, Bill Burke > wrote: > > We have some script to run Mongo? > > On 11/19/2015 7:59 AM, Marek Posolda wrote: > > On 18/11/15 21:11, Stian Thorgersen wrote: > > You need Mongo running on your machine first as the > embedded Mongo > used for the testsuite is ran from Maven. > > Once you have Mongo running locally it's pretty easy, edit > (but don't > commit the changes): > > testsuite/integration/src/test/resources/META-INF/keycloak-server.json > > Change the providers to mongo. There are default settings > for Mongo > already there and should work as long as you have Mongo > running locally. > > I find it easier to just change the keycloak-server.json > directly, but > you can also use system properties to set the providers, > but that's > pretty annoying to do when your running tests from IntelliJ. > > For me, it doesn't seem annoying. I just copy/paste this line with > properties when running any test: > > -Dkeycloak.realm.provider=mongo -Dkeycloak.user.provider=mongo > -Dkeycloak.userSessionPersister.provider=mongo > -Dkeycloak.eventsStore.provider=mongo > -Dkeycloak.connectionsMongo.db=keycloak -Dresources > > I also usually use this for KeycloakServer setup during > development. > > Marek > > > On 18 November 2015 at 17:28, Bill Burke > > >> wrote: > > Anybody know how to run mongo backend with integration > tests within > Intellij? > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/798d9eb6/attachment-0001.html From mposolda at redhat.com Thu Nov 19 08:49:32 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 19 Nov 2015 14:49:32 +0100 Subject: [keycloak-dev] mongo from Intellij? In-Reply-To: References: <564CA50D.7080409@redhat.com> <564CA729.9080401@redhat.com> <564DC7CF.70005@redhat.com> Message-ID: <564DD36C.2070608@redhat.com> Thanks, wasn't aware we have this. Marek On 19/11/15 14:17, Stian Thorgersen wrote: > Actually you can also add those properties to " DIR>/.keycloak-server.properties" as KeycloakServer loads those at > runtime. > > On 19 November 2015 at 13:59, Marek Posolda > wrote: > > On 18/11/15 21:11, Stian Thorgersen wrote: >> You need Mongo running on your machine first as the embedded >> Mongo used for the testsuite is ran from Maven. >> >> Once you have Mongo running locally it's pretty easy, edit (but >> don't commit the changes): >> >> testsuite/integration/src/test/resources/META-INF/keycloak-server.json >> >> Change the providers to mongo. There are default settings for >> Mongo already there and should work as long as you have Mongo >> running locally. >> >> I find it easier to just change the keycloak-server.json >> directly, but you can also use system properties to set the >> providers, but that's pretty annoying to do when your running >> tests from IntelliJ. > For me, it doesn't seem annoying. I just copy/paste this line with > properties when running any test: > > -Dkeycloak.realm.provider=mongo -Dkeycloak.user.provider=mongo > -Dkeycloak.userSessionPersister.provider=mongo > -Dkeycloak.eventsStore.provider=mongo > -Dkeycloak.connectionsMongo.db=keycloak -Dresources > > I also usually use this for KeycloakServer setup during development. > > Marek > >> >> On 18 November 2015 at 17:28, Bill Burke > > wrote: >> >> Anybody know how to run mongo backend with integration tests >> within >> Intellij? >> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/1445622b/attachment.html From mposolda at redhat.com Thu Nov 19 08:55:06 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 19 Nov 2015 14:55:06 +0100 Subject: [keycloak-dev] Change few methods on UserFederationProvider In-Reply-To: <564DD1B0.1070006@redhat.com> References: <564DD111.6090405@redhat.com> <564DD1B0.1070006@redhat.com> Message-ID: <564DD4BA.4080800@redhat.com> Thanks for the confirm. We might have contributor to look at https://issues.jboss.org/browse/KEYCLOAK-1744 . I will ask him to go ahead. Marek On 19/11/15 14:42, Bill Burke wrote: > Sounds good. We're just going to have to be breaking backward > compatibility to clean up these SPIs. Sucks, but necessary. > > On 11/19/2015 8:39 AM, Marek Posolda wrote: >> Currently we have those methods on UserFederationProvider: >> >> boolean validCredentials(RealmModel realm, UserModel user, >> List input); >> boolean validCredentials(RealmModel realm, UserModel user, >> UserCredentialModel... input); >> >> >> I propose if we can: >> 1) Remove the second one as it's not used from anywhere . >> >> 2) Change the signature of first one to return >> "CredentialValidationOutput" instead of boolean. This will allow >> federationProvider to send some additional state related to >> authentication instead of just true/false . >> >> The main reason is the https://issues.jboss.org/browse/KEYCLOAK-1744 . >> Basically ActiveDirectory throws exception with different code if >> password provided by user is incorrect or if the password is correct, >> but expired. For writable LDAP, it's fine. If password is expired, we >> can authenticate user, but put requiredAction for UPDATE_PASSWORD on him. >> >> However for read-only LDAP, we can't update password from Keycloak. In >> this case, it will be nice if we can show the message in UI like "Your >> password has expired. Contact your administrator to change password" . >> But that's possible if we send some additional state about the reason of >> failure, so Authenticator can read it and possibly display various >> messages based on that. >> >> IMO will be cool to have solution for >> https://issues.jboss.org/browse/KEYCLOAK-1744 available in Keycloak out >> of the box. There are lot of people using ActiveDirectory and asking for >> this. >> >> WDYT? >> Marek >> >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> From sthorger at redhat.com Thu Nov 19 08:59:05 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 19 Nov 2015 14:59:05 +0100 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: <564DD34F.603@redhat.com> References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> Message-ID: How about: * IDs are usually generated, but can be specified. It should not be possible to change them though * Instead of alias, name and all other ways of setting a human readable name we add Display Name and Display Class Not sure how to deal with migration, but if we agree on those points, then we can figure that out On 19 November 2015 at 14:49, Bill Burke wrote: > One more thing. Another problem is that if you switch to ID for > everything, then demos etc. will no longer run out of the box and you'll > have to modify adapter configuration to set the newly generated DB id. > Really, any piece of configuration that references things by client ID > will no longer work. > > So, there's really 2 issues: > > * Using ClientID in admin-console URLs doesn't work well as the client > IDs could have '/' characters in them > * Using ID only messes up pre-defined, pre-packaged adapter config and > you have to edit these files after the keycloak DB is set up. > > On 11/19/2015 8:40 AM, Bill Burke wrote: > > We currently actually have 3 client identifers: ID, ClientID, and Name. > > > > Also, I think there are a lot of duplicate clientID names between > > realms. i.e. "admin-console" "broker", etc... > > > > A search by clientID is (realm + clientID). > > > > Marek is right about why I switched everything to ID in the admin > > console. For SAML I just didn't want to add yet another client > > identifier since we already had 3. > > > > On 11/19/2015 7:52 AM, Marek Posolda wrote: > >> +1 for this change. > >> > >> I am just not sure if we should set the "id" to the current value of > >> "client-id" ? Few things to note: > >> > >> - SAML clients currently use clientId in form of URL. For example in > >> SAML demo there are clientIds like "http://localhosT:8080/employee-sig" > >> . I don't know if it's requirement, maybe it's possible to solve it > >> somehow (ie. introduce different attribute for SAML client to store > >> these URLs). But from what I remember, Bill changed admin console to use > >> "id" instead of "clientId" because there were issues with URL-like > >> clientId in admin console . So if we overwrite the "id" with current > >> "client-id" the issue will be back. > >> > >> - Migration might be a pain. Many tables (roles, protocolMappers, user > >> consents, offline clientSessions ...) references client by "id" . > >> Overwriting "id" with "client-id" means that we will need to change all > >> those DB records. And there are things like foreign keys etc... > >> > >> Shouldn't do vice-versa and just remove current "client-id" and ask > >> people to update their keycloak.json adapter configurations? On the > >> other hand, removing "client-id" might break migration of JSON exported > >> realms as the JSON entities are using "client-id" for referencing > client. > >> > >> It seems the migration will be a pain regardless of whatever direction > >> we choose :-( > >> > >> Marek > >> > >> On 16/11/15 14:54, Stian Thorgersen wrote: > >>> We have both "id" and "client-id" for clients in Keycloak at the > >>> moment. This seems unnecessary and complex. > >>> > >>> The model can retrieve clients on either value. In token endpoints the > >>> "client-id" is used. In admin endpoints the "id" is used. > >>> > >>> Also, in most cases it would be simpler for users to just have a > >>> generated id than having to come up with one themselves. The id > >>> doesn't have to be human readable either as we have name for that. > >>> > >>> OpenID Connect expects "client-id" to be generated by the IdP and > >>> can't be changed once created. > >>> > >>> I propose we remove "client-id" and only keep id. > >>> > >>> For migration of existing clients we would set the "id" value to the > >>> current value of "client-id". This would require no changes to adapter > >>> configs. When creating new clients from the admin console we would not > >>> allow setting the "client-id", instead just display it after the > >>> client was created. When importing clients it would be possible to set > >>> the id (and for backwards compatibility we would set "id" equal to the > >>> "client-id" field. > >>> > >>> > >>> _______________________________________________ > >>> keycloak-dev mailing list > >>> keycloak-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > >> > >> > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/1aaf5e76/attachment.html From bburke at redhat.com Thu Nov 19 09:43:04 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 19 Nov 2015 09:43:04 -0500 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> Message-ID: <564DDFF8.9070404@redhat.com> FYI, there's other problems too. Built in clients are looked up via their hardcoded ClientIds "admin-console" "broker" etc, again because DB ids are generated. I just don't see how you can remove ID or ClientID. ID is URL friendly, ClientID isn't and needs to be encoded. ID is pretty much hidden from the user until they want to use the REST interface. If you remove ID from the REST interface then ClientID @PathParams are gonna have to be encoded which could be a real pain for users, especially ones using simple HTTP. While most libraries will encode query and form params, users will have to encode by hand other mechanisms. There's a similar problem with Groups. Groups are looked up via a "path" name "/parent/child" in the REST interface, then you have use the ID to reference the rest of the GROUP rest interface i.e. groups/{ID}/children. We'll have similar problems with role namespaces if you want to incorporate them into the REST interface. ClientId should be forced to become a URL friendly string (no special characters or '/' or "?", etc.). SAML can introduce an Issuer URL or something. After that, you can remove ID. That's the only solution I can see that will work well. On 11/19/2015 8:59 AM, Stian Thorgersen wrote: > How about: > > * IDs are usually generated, but can be specified. It should not be > possible to change them though > > * Instead of alias, name and all other ways of setting a human readable > name we add Display Name and Display Class > > Not sure how to deal with migration, but if we agree on those points, > then we can figure that out > > On 19 November 2015 at 14:49, Bill Burke > wrote: > > One more thing. Another problem is that if you switch to ID for > everything, then demos etc. will no longer run out of the box and you'll > have to modify adapter configuration to set the newly generated DB id. > Really, any piece of configuration that references things by client ID > will no longer work. > > So, there's really 2 issues: > > * Using ClientID in admin-console URLs doesn't work well as the client > IDs could have '/' characters in them > * Using ID only messes up pre-defined, pre-packaged adapter config and > you have to edit these files after the keycloak DB is set up. > > On 11/19/2015 8:40 AM, Bill Burke wrote: > > We currently actually have 3 client identifers: ID, ClientID, > and Name. > > > > Also, I think there are a lot of duplicate clientID names between > > realms. i.e. "admin-console" "broker", etc... > > > > A search by clientID is (realm + clientID). > > > > Marek is right about why I switched everything to ID in the admin > > console. For SAML I just didn't want to add yet another client > > identifier since we already had 3. > > > > On 11/19/2015 7:52 AM, Marek Posolda wrote: > >> +1 for this change. > >> > >> I am just not sure if we should set the "id" to the current value of > >> "client-id" ? Few things to note: > >> > >> - SAML clients currently use clientId in form of URL. For example in > >> SAML demo there are clientIds like > "http://localhosT:8080/employee-sig" > >> . I don't know if it's requirement, maybe it's possible to solve it > >> somehow (ie. introduce different attribute for SAML client to store > >> these URLs). But from what I remember, Bill changed admin > console to use > >> "id" instead of "clientId" because there were issues with URL-like > >> clientId in admin console . So if we overwrite the "id" with current > >> "client-id" the issue will be back. > >> > >> - Migration might be a pain. Many tables (roles, > protocolMappers, user > >> consents, offline clientSessions ...) references client by "id" . > >> Overwriting "id" with "client-id" means that we will need to > change all > >> those DB records. And there are things like foreign keys etc... > >> > >> Shouldn't do vice-versa and just remove current "client-id" and ask > >> people to update their keycloak.json adapter configurations? On the > >> other hand, removing "client-id" might break migration of JSON > exported > >> realms as the JSON entities are using "client-id" for > referencing client. > >> > >> It seems the migration will be a pain regardless of whatever > direction > >> we choose :-( > >> > >> Marek > >> > >> On 16/11/15 14:54, Stian Thorgersen wrote: > >>> We have both "id" and "client-id" for clients in Keycloak at the > >>> moment. This seems unnecessary and complex. > >>> > >>> The model can retrieve clients on either value. In token > endpoints the > >>> "client-id" is used. In admin endpoints the "id" is used. > >>> > >>> Also, in most cases it would be simpler for users to just have a > >>> generated id than having to come up with one themselves. The id > >>> doesn't have to be human readable either as we have name for that. > >>> > >>> OpenID Connect expects "client-id" to be generated by the IdP and > >>> can't be changed once created. > >>> > >>> I propose we remove "client-id" and only keep id. > >>> > >>> For migration of existing clients we would set the "id" value > to the > >>> current value of "client-id". This would require no changes to > adapter > >>> configs. When creating new clients from the admin console we > would not > >>> allow setting the "client-id", instead just display it after the > >>> client was created. When importing clients it would be possible > to set > >>> the id (and for backwards compatibility we would set "id" equal > to the > >>> "client-id" field. > >>> > >>> > >>> _______________________________________________ > >>> keycloak-dev mailing list > >>> keycloak-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > >> > >> > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Thu Nov 19 09:58:43 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 19 Nov 2015 15:58:43 +0100 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: <564DDFF8.9070404@redhat.com> References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> <564DDFF8.9070404@redhat.com> Message-ID: On 19 November 2015 at 15:43, Bill Burke wrote: > FYI, there's other problems too. Built in clients are looked up via their > hardcoded ClientIds "admin-console" "broker" etc, again because DB ids are > generated. > > I just don't see how you can remove ID or ClientID. ID is URL friendly, > ClientID isn't and needs to be encoded. ID is pretty much hidden from the > user until they want to use the REST interface. If you remove ID from the > REST interface then ClientID @PathParams are gonna have to be encoded which > could be a real pain for users, especially ones using simple HTTP. While > most libraries will encode query and form params, users will have to encode > by hand other mechanisms. > For clients I was thinking we should remove ClientID, not ID. We'd allow specifying the ID when creating a new client, but not changing it. The ID would also have to be URL friendly. > > There's a similar problem with Groups. Groups are looked up via a "path" > name "/parent/child" in the REST interface, then you have use the ID to > reference the rest of the GROUP rest interface i.e. groups/{ID}/children. > > We'll have similar problems with role namespaces if you want to > incorporate them into the REST interface. > > ClientId should be forced to become a URL friendly string (no special > characters or '/' or "?", etc.). SAML can introduce an Issuer URL or > something. After that, you can remove ID. That's the only solution I can > see that will work well. For groups and roles couldn't the ID be the full path? Then the REST endpoints would be aware of "hierarchies" so you could just do: get /auth/realms/groups/group-parent/group-child And: get /auth/realms/roles/role/path/to/role/myrole ?? I think the current approach of having REST endpoints that retrieve based on ID is a pain point for a lot of users. They create a client with a named client-id or role with a name, then have to use a different ID to retrieve from the rest endpoints > > > > On 11/19/2015 8:59 AM, Stian Thorgersen wrote: > >> How about: >> >> * IDs are usually generated, but can be specified. It should not be >> possible to change them though >> >> * Instead of alias, name and all other ways of setting a human readable >> name we add Display Name and Display Class >> >> Not sure how to deal with migration, but if we agree on those points, >> then we can figure that out >> >> On 19 November 2015 at 14:49, Bill Burke > > wrote: >> >> One more thing. Another problem is that if you switch to ID for >> everything, then demos etc. will no longer run out of the box and >> you'll >> have to modify adapter configuration to set the newly generated DB id. >> Really, any piece of configuration that references things by client ID >> will no longer work. >> >> So, there's really 2 issues: >> >> * Using ClientID in admin-console URLs doesn't work well as the client >> IDs could have '/' characters in them >> * Using ID only messes up pre-defined, pre-packaged adapter config and >> you have to edit these files after the keycloak DB is set up. >> >> On 11/19/2015 8:40 AM, Bill Burke wrote: >> > We currently actually have 3 client identifers: ID, ClientID, >> and Name. >> > >> > Also, I think there are a lot of duplicate clientID names between >> > realms. i.e. "admin-console" "broker", etc... >> > >> > A search by clientID is (realm + clientID). >> > >> > Marek is right about why I switched everything to ID in the admin >> > console. For SAML I just didn't want to add yet another client >> > identifier since we already had 3. >> > >> > On 11/19/2015 7:52 AM, Marek Posolda wrote: >> >> +1 for this change. >> >> >> >> I am just not sure if we should set the "id" to the current value >> of >> >> "client-id" ? Few things to note: >> >> >> >> - SAML clients currently use clientId in form of URL. For example >> in >> >> SAML demo there are clientIds like >> "http://localhosT:8080/employee-sig" >> >> . I don't know if it's requirement, maybe it's possible to solve >> it >> >> somehow (ie. introduce different attribute for SAML client to >> store >> >> these URLs). But from what I remember, Bill changed admin >> console to use >> >> "id" instead of "clientId" because there were issues with URL-like >> >> clientId in admin console . So if we overwrite the "id" with >> current >> >> "client-id" the issue will be back. >> >> >> >> - Migration might be a pain. Many tables (roles, >> protocolMappers, user >> >> consents, offline clientSessions ...) references client by "id" . >> >> Overwriting "id" with "client-id" means that we will need to >> change all >> >> those DB records. And there are things like foreign keys etc... >> >> >> >> Shouldn't do vice-versa and just remove current "client-id" and >> ask >> >> people to update their keycloak.json adapter configurations? On >> the >> >> other hand, removing "client-id" might break migration of JSON >> exported >> >> realms as the JSON entities are using "client-id" for >> referencing client. >> >> >> >> It seems the migration will be a pain regardless of whatever >> direction >> >> we choose :-( >> >> >> >> Marek >> >> >> >> On 16/11/15 14:54, Stian Thorgersen wrote: >> >>> We have both "id" and "client-id" for clients in Keycloak at the >> >>> moment. This seems unnecessary and complex. >> >>> >> >>> The model can retrieve clients on either value. In token >> endpoints the >> >>> "client-id" is used. In admin endpoints the "id" is used. >> >>> >> >>> Also, in most cases it would be simpler for users to just have a >> >>> generated id than having to come up with one themselves. The id >> >>> doesn't have to be human readable either as we have name for >> that. >> >>> >> >>> OpenID Connect expects "client-id" to be generated by the IdP and >> >>> can't be changed once created. >> >>> >> >>> I propose we remove "client-id" and only keep id. >> >>> >> >>> For migration of existing clients we would set the "id" value >> to the >> >>> current value of "client-id". This would require no changes to >> adapter >> >>> configs. When creating new clients from the admin console we >> would not >> >>> allow setting the "client-id", instead just display it after the >> >>> client was created. When importing clients it would be possible >> to set >> >>> the id (and for backwards compatibility we would set "id" equal >> to the >> >>> "client-id" field. >> >>> >> >>> >> >>> _______________________________________________ >> >>> keycloak-dev mailing list >> >>> keycloak-dev at lists.jboss.org > keycloak-dev at lists.jboss.org> >> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> >> >> >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev at lists.jboss.org > > >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/2ae60342/attachment.html From mposolda at redhat.com Thu Nov 19 10:54:22 2015 From: mposolda at redhat.com (Marek Posolda) Date: Thu, 19 Nov 2015 16:54:22 +0100 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> Message-ID: <564DF0AE.5050809@redhat.com> On 19/11/15 14:59, Stian Thorgersen wrote: > How about: > > * IDs are usually generated, but can be specified. It should not be > possible to change them though > > * Instead of alias, name and all other ways of setting a human > readable name we add Display Name and Display Class So this will be for all other objects, not just clients? Again +1 as currently it's mess. Various objects have various "names" for kind of same thing. Or even more they have same names, for different things. For example: - IdentityProviderModel has: internalId + alias - UserFederationModel has: id + displayName - role has: id + name + description (Name is unchangeable and not localized, so have different meaning then "name" for client etc) But not sure what is "Display Class" . Is it the same thing like "description" ? I would personally use: - ID - displayName - description > > Not sure how to deal with migration, but if we agree on those points, > then we can figure that out > > On 19 November 2015 at 14:49, Bill Burke > wrote: > > One more thing. Another problem is that if you switch to ID for > everything, then demos etc. will no longer run out of the box and > you'll > have to modify adapter configuration to set the newly generated DB id. > Really, any piece of configuration that references things by client ID > will no longer work. > > So, there's really 2 issues: > > * Using ClientID in admin-console URLs doesn't work well as the client > IDs could have '/' characters in them > * Using ID only messes up pre-defined, pre-packaged adapter config and > you have to edit these files after the keycloak DB is set up. > > On 11/19/2015 8:40 AM, Bill Burke wrote: > > We currently actually have 3 client identifers: ID, ClientID, > and Name. > > > > Also, I think there are a lot of duplicate clientID names between > > realms. i.e. "admin-console" "broker", etc... > > > > A search by clientID is (realm + clientID). > > > > Marek is right about why I switched everything to ID in the admin > > console. For SAML I just didn't want to add yet another client > > identifier since we already had 3. > > > > On 11/19/2015 7:52 AM, Marek Posolda wrote: > >> +1 for this change. > >> > >> I am just not sure if we should set the "id" to the current > value of > >> "client-id" ? Few things to note: > >> > >> - SAML clients currently use clientId in form of URL. For > example in > >> SAML demo there are clientIds like > "http://localhosT:8080/employee-sig" > >> . I don't know if it's requirement, maybe it's possible to solve it > >> somehow (ie. introduce different attribute for SAML client to store > >> these URLs). But from what I remember, Bill changed admin > console to use > >> "id" instead of "clientId" because there were issues with > URL-likeadmin > >> clientId in admin console . So if we overwrite the "id" with > current > >> "client-id" the issue will be back. > >> > >> - Migration might be a pain. Many tables (roles, > protocolMappers, user > >> consents, offline clientSessions ...) references client by "id" . > >> Overwriting "id" with "client-id" means that we will need to > change all > >> those DB records. And there are things like foreign keys etc... > >> > >> Shouldn't do vice-versa and just remove current "client-id" and ask > >> people to update their keycloak.json adapter configurations? On the > >> other hand, removing "client-id" might break migration of JSON > exported > >> realms as the JSON entities are using "client-id" for > referencing client. > >> > >> It seems the migration will be a pain regardless of whatever > direction > >> we choose :-( > >> > >> Marek > >> > >> On 16/11/15 14:54, Stian Thorgersen wrote: > >>> We have both "id" and "client-id" for clients in Keycloak at the > >>> moment. This seems unnecessary and complex. > >>> > >>> The model can retrieve clients on either value. In token > endpoints the > >>> "client-id" is used. In admin endpoints the "id" is used. > >>> > >>> Also, in most cases it would be simpler for users to just have a > >>> generated id than having to come up with one themselves. The id > >>> doesn't have to be human readable either as we have name for that. > >>> > >>> OpenID Connect expects "client-id" to be generated by the IdP and > >>> can't be changed once created. > >>> > >>> I propose we remove "client-id" and only keep id. > >>> > >>> For migration of existing clients we would set the "id" value > to the > >>> current value of "client-id". This would require no changes to > adapter > >>> configs. When creating new clients from the admin console we > would not > >>> allow setting the "client-id", instead just display it after the > >>> client was created. When importing clients it would be > possible to set > >>> the id (and for backwards compatibility we would set "id" > equal to the > >>> "client-id" field. > >>> > >>> > >>> _______________________________________________ > >>> keycloak-dev mailing list > >>> keycloak-dev at lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > >> > >> > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/1fbb36f1/attachment-0001.html From bburke at redhat.com Thu Nov 19 13:26:09 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 19 Nov 2015 13:26:09 -0500 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> <564DDFF8.9070404@redhat.com> Message-ID: <564E1441.4000604@redhat.com> On 11/19/2015 9:58 AM, Stian Thorgersen wrote: > > > On 19 November 2015 at 15:43, Bill Burke > wrote: > > FYI, there's other problems too. Built in clients are looked up via > their hardcoded ClientIds "admin-console" "broker" etc, again > because DB ids are generated. > > I just don't see how you can remove ID or ClientID. ID is URL > friendly, ClientID isn't and needs to be encoded. ID is pretty much > hidden from the user until they want to use the REST interface. If > you remove ID from the REST interface then ClientID @PathParams are > gonna have to be encoded which could be a real pain for users, > especially ones using simple HTTP. While most libraries will encode > query and form params, users will have to encode by hand other > mechanisms. > > > For clients I was thinking we should remove ClientID, not ID. We'd allow > specifying the ID when creating a new client, but not changing it. The > ID would also have to be URL friendly. > Removing clientID doesn't work for built-in clients like account, broker, admin-console, etc. These all need to be located using a predetermined name. You'd have to figure out an additional alternative to refactor that. > > There's a similar problem with Groups. Groups are looked up via a > "path" name "/parent/child" in the REST interface, then you have > use the ID to reference the rest of the GROUP rest interface i.e. > groups/{ID}/children. > > > We'll have similar problems with role namespaces if you want to > incorporate them into the REST interface. > > ClientId should be forced to become a URL friendly string (no > special characters or '/' or "?", etc.). SAML can introduce an > Issuer URL or something. After that, you can remove ID. That's the > only solution I can see that will work well. > > > For groups and roles couldn't the ID be the full path? Then the REST > endpoints would be aware of "hierarchies" so you could just do: > > get /auth/realms/groups/group-parent/group-child > > And: > > get /auth/realms/roles/role/path/to/role/myrole > I didn't even try to see if that worked. The issue is that there are additional endpoints after the path: .../children .../composites And methods to add remove group children and role composites. I'm pretty sure there are some JAX-RS and/or regular-expression mapping issues with that. At least that's what I remember, might be a false memory. > ?? > > I think the current approach of having REST endpoints that retrieve > based on ID is a pain point for a lot of users. They create a client > with a named client-id or role with a name, then have to use a different > ID to retrieve from the rest endpoints > The way it works for groups now is that i have a /group-by-path lookup mechanism which returns a GroupRepresentation, the id is extracted from that, then you invoke from there, i.e.: GroupRepresentation topGroup = realm.getGroupByPath("/the/top"); List roles = new LinkedList<>(); roles.add(topRole); realm.groups().group(topGroup.getId()).roles().realmLevel().add(roles); Alternatively we could add links to the GroupRepresentation like 'self' that points to the resource endpoint. > > > > On 11/19/2015 8:59 AM, Stian Thorgersen wrote: > > How about: > > * IDs are usually generated, but can be specified. It should not be > possible to change them though > > * Instead of alias, name and all other ways of setting a human > readable > name we add Display Name and Display Class > > Not sure how to deal with migration, but if we agree on those > points, > then we can figure that out > > On 19 November 2015 at 14:49, Bill Burke > >> wrote: > > One more thing. Another problem is that if you switch to > ID for > everything, then demos etc. will no longer run out of the > box and you'll > have to modify adapter configuration to set the newly > generated DB id. > Really, any piece of configuration that references things > by client ID > will no longer work. > > So, there's really 2 issues: > > * Using ClientID in admin-console URLs doesn't work well as > the client > IDs could have '/' characters in them > * Using ID only messes up pre-defined, pre-packaged adapter > config and > you have to edit these files after the keycloak DB is set up. > > On 11/19/2015 8:40 AM, Bill Burke wrote: > > We currently actually have 3 client identifers: ID, > ClientID, > and Name. > > > > Also, I think there are a lot of duplicate clientID > names between > > realms. i.e. "admin-console" "broker", etc... > > > > A search by clientID is (realm + clientID). > > > > Marek is right about why I switched everything to ID in > the admin > > console. For SAML I just didn't want to add yet another > client > > identifier since we already had 3. > > > > On 11/19/2015 7:52 AM, Marek Posolda wrote: > >> +1 for this change. > >> > >> I am just not sure if we should set the "id" to the > current value of > >> "client-id" ? Few things to note: > >> > >> - SAML clients currently use clientId in form of URL. > For example in > >> SAML demo there are clientIds like > "http://localhosT:8080/employee-sig" > >> . I don't know if it's requirement, maybe it's possible > to solve it > >> somehow (ie. introduce different attribute for SAML > client to store > >> these URLs). But from what I remember, Bill changed admin > console to use > >> "id" instead of "clientId" because there were issues > with URL-like > >> clientId in admin console . So if we overwrite the "id" > with current > >> "client-id" the issue will be back. > >> > >> - Migration might be a pain. Many tables (roles, > protocolMappers, user > >> consents, offline clientSessions ...) references client > by "id" . > >> Overwriting "id" with "client-id" means that we will > need to > change all > >> those DB records. And there are things like foreign > keys etc... > >> > >> Shouldn't do vice-versa and just remove current > "client-id" and ask > >> people to update their keycloak.json adapter > configurations? On the > >> other hand, removing "client-id" might break migration > of JSON > exported > >> realms as the JSON entities are using "client-id" for > referencing client. > >> > >> It seems the migration will be a pain regardless of > whatever > direction > >> we choose :-( > >> > >> Marek > >> > >> On 16/11/15 14:54, Stian Thorgersen wrote: > >>> We have both "id" and "client-id" for clients in > Keycloak at the > >>> moment. This seems unnecessary and complex. > >>> > >>> The model can retrieve clients on either value. In token > endpoints the > >>> "client-id" is used. In admin endpoints the "id" is used. > >>> > >>> Also, in most cases it would be simpler for users to > just have a > >>> generated id than having to come up with one > themselves. The id > >>> doesn't have to be human readable either as we have > name for that. > >>> > >>> OpenID Connect expects "client-id" to be generated by > the IdP and > >>> can't be changed once created. > >>> > >>> I propose we remove "client-id" and only keep id. > >>> > >>> For migration of existing clients we would set the > "id" value > to the > >>> current value of "client-id". This would require no > changes to > adapter > >>> configs. When creating new clients from the admin > console we > would not > >>> allow setting the "client-id", instead just display it > after the > >>> client was created. When importing clients it would be > possible > to set > >>> the id (and for backwards compatibility we would set > "id" equal > to the > >>> "client-id" field. > >>> > >>> > >>> _______________________________________________ > >>> keycloak-dev mailing list > >>> keycloak-dev at lists.jboss.org > > > > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > >> > >> > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > > > > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >> > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > > > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From Dane.Barentine at software.dell.com Thu Nov 19 18:06:06 2015 From: Dane.Barentine at software.dell.com (Dane Barentine) Date: Thu, 19 Nov 2015 23:06:06 +0000 Subject: [keycloak-dev] Identity Broker login flow Message-ID: Hi all, I'm trying to add a custom authenticator and it appears that that there is no way to insert it in the flow if it's a brokered IDP login where the linked Keycloak account already exists. If it's a local Keycloak user I can use the Browser flow and if it's a new brokered user the First Broker Login flow will execute. But I don't see a flow that would allow me to insert something like OTP after a brokered login of an existing user. If I'm just missing it let me know but I think there needs to be some sort of flow for brokered logins that runs on both existing and new users. For new users it would run after the First Broker Login flow. Or better yet maybe a flow that would allow things such as OTP to happen after any brokered or local login. That way it wouldn't have to be configured in multiple flows. Thanks Dane -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151119/90a0df89/attachment.html From mposolda at redhat.com Fri Nov 20 02:41:43 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 20 Nov 2015 08:41:43 +0100 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: References: Message-ID: <564ECEB7.3020401@redhat.com> You're right, we don't have this right now. I don't know if it's something we should support OOTB. The idea of broker login is, that you delegate authentication to another SSO/social server. Once the second server say "Ok, user is authenticated", we treat him as authenticated on Keycloak side too. Isn't it the more proper option for your usecase to use OTP on the second server side instead? Another option is to implement IdentityProviderMapper and in "updateBrokeredUser" method, you will redirect user to OTP login. Could you try this? If we want to support another login flow triggered after each broker login (which I am not convinced TBH), we can either: 1) Introduce "post-broker-login" flow, which will be configurable per IdentityProvider. By default, it will be empty . 2) Use just one flow, which will be triggered after each broker login (current "first broker login" flow is triggered only if federatedIdentity doesn't yet exist in Keycloak). In this case, the current "first broker login" flow will need to be renamed to "broker login" and more logic will need to be moved from IdentityBrokerService to the flow itself. The disadvantage of this option is, that it may always require another redirect to trigger authentication flows. But we're trying to reduce the number of redirects ( https://issues.jboss.org/browse/KEYCLOAK-2098 ) Marek On 20/11/15 00:06, Dane Barentine wrote: > > Hi all, > > I?m trying to add a custom authenticator and it appears that that > there is no way to insert it in the flow if it?s a brokered IDP login > where the linked Keycloak account already exists. > > If it?s a local Keycloak user I can use the Browser flow and if it?s a > new brokered user the First Broker Login flow will execute. But I > don?t see a flow that would allow me to insert something like OTP > after a brokered login of an existing user. > > If I?m just missing it let me know but I think there needs to be some > sort of flow for brokered logins that runs on both existing and new > users. For new users it would run after the First Broker Login flow. > Or better yet maybe a flow that would allow things such as OTP to > happen after any brokered or local login. That way it wouldn?t have to > be configured in multiple flows. > > Thanks > > Dane > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151120/855cb8c9/attachment.html From mposolda at redhat.com Fri Nov 20 03:45:09 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 20 Nov 2015 09:45:09 +0100 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: <564ECEB7.3020401@redhat.com> References: <564ECEB7.3020401@redhat.com> Message-ID: <564EDD95.4090101@redhat.com> I wonder if we should address this together with reducing number of redirects ( https://issues.jboss.org/browse/KEYCLOAK-2098 ) and also support for "authentication levels" . We can encapsulate all the state of authentication in ClientSession. If someone refreshes the page, we will retrieve the ClientSession from the code and the current state of execution will be retrieved from ClientSession itself. ClientSession will also encapsulate all the state of authentication after authentication is finished. For example there will be "cookie" if ClientSession was authenticated through cookie, "password, otp" if clientSession was authenticated through password + otp or "broker-facebook" if it was authenticated through Facebook. There will be also timestamps of when each successful authentication happened. The state will be partially saved in UserSession, so if clientSession was authenticated through cookie, it can retrieve from userSession when the "real" authentication happened etc. This will allow us to support periodic re-authentication, authentication levels and different requirements for "authentication level" for individual application. For example at each SSO login or refresh, we will be able to: - ask user for re-authenticate through OTP every 30 minutes - ask user for re-authenticate through OTP if he was authenticated through Facebook broker - ask user for re-authenticate through OTP if application required that with some magic parameter (AFAIK both OIDC or SAML has some parameters for specify authentication levels) Marek On 20/11/15 08:41, Marek Posolda wrote: > You're right, we don't have this right now. I don't know if it's > something we should support OOTB. The idea of broker login is, that > you delegate authentication to another SSO/social server. Once the > second server say "Ok, user is authenticated", we treat him as > authenticated on Keycloak side too. Isn't it the more proper option > for your usecase to use OTP on the second server side instead? > > Another option is to implement IdentityProviderMapper and in > "updateBrokeredUser" method, you will redirect user to OTP login. > Could you try this? > > > If we want to support another login flow triggered after each broker > login (which I am not convinced TBH), we can either: > > 1) Introduce "post-broker-login" flow, which will be configurable per > IdentityProvider. By default, it will be empty . > > 2) Use just one flow, which will be triggered after each broker login > (current "first broker login" flow is triggered only if > federatedIdentity doesn't yet exist in Keycloak). In this case, the > current "first broker login" flow will need to be renamed to "broker > login" and more logic will need to be moved from IdentityBrokerService > to the flow itself. The disadvantage of this option is, that it may > always require another redirect to trigger authentication flows. But > we're trying to reduce the number of redirects ( > https://issues.jboss.org/browse/KEYCLOAK-2098 ) > > Marek > > On 20/11/15 00:06, Dane Barentine wrote: >> >> Hi all, >> >> I?m trying to add a custom authenticator and it appears that that >> there is no way to insert it in the flow if it?s a brokered IDP login >> where the linked Keycloak account already exists. >> >> If it?s a local Keycloak user I can use the Browser flow and if it?s >> a new brokered user the First Broker Login flow will execute. But I >> don?t see a flow that would allow me to insert something like OTP >> after a brokered login of an existing user. >> >> If I?m just missing it let me know but I think there needs to be some >> sort of flow for brokered logins that runs on both existing and new >> users. For new users it would run after the First Broker Login flow. >> Or better yet maybe a flow that would allow things such as OTP to >> happen after any brokered or local login. That way it wouldn?t have >> to be configured in multiple flows. >> >> Thanks >> >> Dane >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151120/c5c8c24e/attachment-0001.html From gerbermichi at me.com Fri Nov 20 09:04:56 2015 From: gerbermichi at me.com (Michael Gerber) Date: Fri, 20 Nov 2015 14:04:56 +0000 (GMT) Subject: [keycloak-dev] no empty password in UserFederationProvider Message-ID: Hi All, keycloak does not pass an empty password to the?validCredentials method in the?UserFederationProvider class. Is there a reason for that? I would like?to authenticate against an AD even if the password is empty, otherwise the user won't be blocked after x attempts. Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151120/6ae4a48b/attachment.html From bburke at redhat.com Fri Nov 20 10:05:49 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 20 Nov 2015 10:05:49 -0500 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: <564EDD95.4090101@redhat.com> References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> Message-ID: <564F36CD.5000003@redhat.com> All something we want to do, but do we have time? Very soon we need to start focusing on cleaning up the APIs/SPIs, documentation, improving UI, performance, and the tons of minor problems people are experiencing. On 11/20/2015 3:45 AM, Marek Posolda wrote: > I wonder if we should address this together with reducing number of > redirects ( https://issues.jboss.org/browse/KEYCLOAK-2098 ) and also > support for "authentication levels" . We can encapsulate all the state > of authentication in ClientSession. If someone refreshes the page, we > will retrieve the ClientSession from the code and the current state of > execution will be retrieved from ClientSession itself. > > ClientSession will also encapsulate all the state of authentication > after authentication is finished. For example there will be "cookie" if > ClientSession was authenticated through cookie, "password, otp" if > clientSession was authenticated through password + otp or > "broker-facebook" if it was authenticated through Facebook. There will > be also timestamps of when each successful authentication happened. The > state will be partially saved in UserSession, so if clientSession was > authenticated through cookie, it can retrieve from userSession when the > "real" authentication happened etc. > > This will allow us to support periodic re-authentication, authentication > levels and different requirements for "authentication level" for > individual application. For example at each SSO login or refresh, we > will be able to: > - ask user for re-authenticate through OTP every 30 minutes > - ask user for re-authenticate through OTP if he was authenticated > through Facebook broker > - ask user for re-authenticate through OTP if application required that > with some magic parameter (AFAIK both OIDC or SAML has some parameters > for specify authentication levels) > > Marek > > On 20/11/15 08:41, Marek Posolda wrote: >> You're right, we don't have this right now. I don't know if it's >> something we should support OOTB. The idea of broker login is, that >> you delegate authentication to another SSO/social server. Once the >> second server say "Ok, user is authenticated", we treat him as >> authenticated on Keycloak side too. Isn't it the more proper option >> for your usecase to use OTP on the second server side instead? >> >> Another option is to implement IdentityProviderMapper and in >> "updateBrokeredUser" method, you will redirect user to OTP login. >> Could you try this? >> >> >> If we want to support another login flow triggered after each broker >> login (which I am not convinced TBH), we can either: >> >> 1) Introduce "post-broker-login" flow, which will be configurable per >> IdentityProvider. By default, it will be empty . >> >> 2) Use just one flow, which will be triggered after each broker login >> (current "first broker login" flow is triggered only if >> federatedIdentity doesn't yet exist in Keycloak). In this case, the >> current "first broker login" flow will need to be renamed to "broker >> login" and more logic will need to be moved from IdentityBrokerService >> to the flow itself. The disadvantage of this option is, that it may >> always require another redirect to trigger authentication flows. But >> we're trying to reduce the number of redirects ( >> https://issues.jboss.org/browse/KEYCLOAK-2098 ) >> >> Marek >> >> On 20/11/15 00:06, Dane Barentine wrote: >>> >>> Hi all, >>> >>> I?m trying to add a custom authenticator and it appears that that >>> there is no way to insert it in the flow if it?s a brokered IDP login >>> where the linked Keycloak account already exists. >>> >>> If it?s a local Keycloak user I can use the Browser flow and if it?s >>> a new brokered user the First Broker Login flow will execute. But I >>> don?t see a flow that would allow me to insert something like OTP >>> after a brokered login of an existing user. >>> >>> If I?m just missing it let me know but I think there needs to be some >>> sort of flow for brokered logins that runs on both existing and new >>> users. For new users it would run after the First Broker Login flow. >>> Or better yet maybe a flow that would allow things such as OTP to >>> happen after any brokered or local login. That way it wouldn?t have >>> to be configured in multiple flows. >>> >>> Thanks >>> >>> Dane >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Fri Nov 20 10:11:09 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 20 Nov 2015 10:11:09 -0500 Subject: [keycloak-dev] no empty password in UserFederationProvider In-Reply-To: References: Message-ID: <564F380D.2070505@redhat.com> Point me to the code? On 11/20/2015 9:04 AM, Michael Gerber wrote: > Hi All, > > keycloak does not pass an empty password to the validCredentials method > in the UserFederationProvider class. > Is there a reason for that? I would like to authenticate against an AD > even if the password is empty, otherwise the user won't be blocked after > x attempts. > > Michael > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From gerbermichi at me.com Fri Nov 20 10:16:26 2015 From: gerbermichi at me.com (Michael Gerber) Date: Fri, 20 Nov 2015 15:16:26 +0000 (GMT) Subject: [keycloak-dev] =?utf-8?q?_Re=3A__no_empty_password_in_UserFederat?= =?utf-8?q?ionProvider?= Message-ID: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> AbstractUsernameFormAuthenticator.validatePassword public boolean validatePassword(AuthenticationFlowContext context, UserModel user, MultivaluedMap inputData) { List credentials = new LinkedList<>(); String password = inputData.getFirst(CredentialRepresentation.PASSWORD); if (password == null || password.isEmpty()) { invalidPassword(context, user); return false; } credentials.add(UserCredentialModel.password(password)); boolean valid = context.getSession().users().validCredentials(context.getRealm(), user, credentials); if (!valid) { invalidPassword(context, user); return false; } return true; } I think we can remove the first if (password == null || password.isEmpty()) Am 20. November 2015 um 16:11 schrieb Bill Burke : Point me to the code? On 11/20/2015 9:04 AM, Michael Gerber wrote: Hi All, keycloak does not pass an empty password to the validCredentials method in the UserFederationProvider class. Is there a reason for that? I would like to authenticate against an AD even if the password is empty, otherwise the user won't be blocked after x attempts. Michael _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151120/fd95d472/attachment.html From mposolda at redhat.com Fri Nov 20 11:18:44 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 20 Nov 2015 17:18:44 +0100 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: <564F36CD.5000003@redhat.com> References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> Message-ID: <564F47E4.3010809@redhat.com> Yep. I know :-\ As usually, the most proper solution requires most of the changes and refactoring... For this particular case, I wonder if Dane can use the approach with implement IdentityProviderMapper to redirect to OTP login. This is workaround though (IdentityProviderMapper is likely not designed for the usecases like this), however I think it should work. This won't require any changes in existing Keycloak code. I can introduce "post-broker-login" flow, but not sure if it's something we should support OOTB. I don't know if it's useful for more people. So I would rather skip adding post-broker-login as "authentication levels" will be more proper and more generic solution IMO. Marek On 20/11/15 16:05, Bill Burke wrote: > All something we want to do, but do we have time? Very soon we need to > start focusing on cleaning up the APIs/SPIs, documentation, improving > UI, performance, and the tons of minor problems people are experiencing. > > On 11/20/2015 3:45 AM, Marek Posolda wrote: >> I wonder if we should address this together with reducing number of >> redirects ( https://issues.jboss.org/browse/KEYCLOAK-2098 ) and also >> support for "authentication levels" . We can encapsulate all the state >> of authentication in ClientSession. If someone refreshes the page, we >> will retrieve the ClientSession from the code and the current state of >> execution will be retrieved from ClientSession itself. >> >> ClientSession will also encapsulate all the state of authentication >> after authentication is finished. For example there will be "cookie" if >> ClientSession was authenticated through cookie, "password, otp" if >> clientSession was authenticated through password + otp or >> "broker-facebook" if it was authenticated through Facebook. There will >> be also timestamps of when each successful authentication happened. The >> state will be partially saved in UserSession, so if clientSession was >> authenticated through cookie, it can retrieve from userSession when the >> "real" authentication happened etc. >> >> This will allow us to support periodic re-authentication, authentication >> levels and different requirements for "authentication level" for >> individual application. For example at each SSO login or refresh, we >> will be able to: >> - ask user for re-authenticate through OTP every 30 minutes >> - ask user for re-authenticate through OTP if he was authenticated >> through Facebook broker >> - ask user for re-authenticate through OTP if application required that >> with some magic parameter (AFAIK both OIDC or SAML has some parameters >> for specify authentication levels) >> >> Marek >> >> On 20/11/15 08:41, Marek Posolda wrote: >>> You're right, we don't have this right now. I don't know if it's >>> something we should support OOTB. The idea of broker login is, that >>> you delegate authentication to another SSO/social server. Once the >>> second server say "Ok, user is authenticated", we treat him as >>> authenticated on Keycloak side too. Isn't it the more proper option >>> for your usecase to use OTP on the second server side instead? >>> >>> Another option is to implement IdentityProviderMapper and in >>> "updateBrokeredUser" method, you will redirect user to OTP login. >>> Could you try this? >>> >>> >>> If we want to support another login flow triggered after each broker >>> login (which I am not convinced TBH), we can either: >>> >>> 1) Introduce "post-broker-login" flow, which will be configurable per >>> IdentityProvider. By default, it will be empty . >>> >>> 2) Use just one flow, which will be triggered after each broker login >>> (current "first broker login" flow is triggered only if >>> federatedIdentity doesn't yet exist in Keycloak). In this case, the >>> current "first broker login" flow will need to be renamed to "broker >>> login" and more logic will need to be moved from IdentityBrokerService >>> to the flow itself. The disadvantage of this option is, that it may >>> always require another redirect to trigger authentication flows. But >>> we're trying to reduce the number of redirects ( >>> https://issues.jboss.org/browse/KEYCLOAK-2098 ) >>> >>> Marek >>> >>> On 20/11/15 00:06, Dane Barentine wrote: >>>> Hi all, >>>> >>>> I?m trying to add a custom authenticator and it appears that that >>>> there is no way to insert it in the flow if it?s a brokered IDP login >>>> where the linked Keycloak account already exists. >>>> >>>> If it?s a local Keycloak user I can use the Browser flow and if it?s >>>> a new brokered user the First Broker Login flow will execute. But I >>>> don?t see a flow that would allow me to insert something like OTP >>>> after a brokered login of an existing user. >>>> >>>> If I?m just missing it let me know but I think there needs to be some >>>> sort of flow for brokered logins that runs on both existing and new >>>> users. For new users it would run after the First Broker Login flow. >>>> Or better yet maybe a flow that would allow things such as OTP to >>>> happen after any brokered or local login. That way it wouldn?t have >>>> to be configured in multiple flows. >>>> >>>> Thanks >>>> >>>> Dane >>>> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> From bburke at redhat.com Fri Nov 20 11:40:38 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 20 Nov 2015 11:40:38 -0500 Subject: [keycloak-dev] no empty password in UserFederationProvider In-Reply-To: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> References: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> Message-ID: <564F4D06.2070205@redhat.com> You can I guess, but why does it matter? invalidPassword hits the brute force detector if it is turned on. On 11/20/2015 10:16 AM, Michael Gerber wrote: > AbstractUsernameFormAuthenticator.validatePassword > > public boolean validatePassword(AuthenticationFlowContext context, UserModel user, MultivaluedMap inputData) { > List credentials =new LinkedList<>(); > String password = inputData.getFirst(CredentialRepresentation.PASSWORD); > if (password ==null || password.isEmpty()) { > invalidPassword(context, user); > return false; > } > credentials.add(UserCredentialModel.password(password)); > boolean valid = context.getSession().users().validCredentials(context.getRealm(), user, credentials); > if (!valid) { > invalidPassword(context, user); > return false; > } > return true; > } > > I think we can remove the first if (password == null || password.isEmpty()) > > Am 20. November 2015 um 16:11 schrieb Bill Burke : > >> Point me to the code? >> >> On 11/20/2015 9:04 AM, Michael Gerber wrote: >>> Hi All, >>> >>> keycloak does not pass an empty password to the validCredentials method >>> in the UserFederationProvider class. >>> Is there a reason for that? I would like to authenticate against an AD >>> even if the password is empty, otherwise the user won't be blocked after >>> x attempts. >>> >>> Michael >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Fri Nov 20 11:41:46 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 20 Nov 2015 11:41:46 -0500 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: <564F47E4.3010809@redhat.com> References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> <564F47E4.3010809@redhat.com> Message-ID: <564F4D4A.3050805@redhat.com> Oh, I thought our broker flows happened after the user logged into the external IDP? On 11/20/2015 11:18 AM, Marek Posolda wrote: > Yep. I know :-\ > > As usually, the most proper solution requires most of the changes and > refactoring... > > For this particular case, I wonder if Dane can use the approach with > implement IdentityProviderMapper to redirect to OTP login. This is > workaround though (IdentityProviderMapper is likely not designed for the > usecases like this), however I think it should work. This won't require > any changes in existing Keycloak code. > > I can introduce "post-broker-login" flow, but not sure if it's something > we should support OOTB. I don't know if it's useful for more people. So > I would rather skip adding post-broker-login as "authentication levels" > will be more proper and more generic solution IMO. > > Marek > > > On 20/11/15 16:05, Bill Burke wrote: >> All something we want to do, but do we have time? Very soon we need to >> start focusing on cleaning up the APIs/SPIs, documentation, improving >> UI, performance, and the tons of minor problems people are experiencing. >> >> On 11/20/2015 3:45 AM, Marek Posolda wrote: >>> I wonder if we should address this together with reducing number of >>> redirects ( https://issues.jboss.org/browse/KEYCLOAK-2098 ) and also >>> support for "authentication levels" . We can encapsulate all the state >>> of authentication in ClientSession. If someone refreshes the page, we >>> will retrieve the ClientSession from the code and the current state of >>> execution will be retrieved from ClientSession itself. >>> >>> ClientSession will also encapsulate all the state of authentication >>> after authentication is finished. For example there will be "cookie" if >>> ClientSession was authenticated through cookie, "password, otp" if >>> clientSession was authenticated through password + otp or >>> "broker-facebook" if it was authenticated through Facebook. There will >>> be also timestamps of when each successful authentication happened. The >>> state will be partially saved in UserSession, so if clientSession was >>> authenticated through cookie, it can retrieve from userSession when the >>> "real" authentication happened etc. >>> >>> This will allow us to support periodic re-authentication, authentication >>> levels and different requirements for "authentication level" for >>> individual application. For example at each SSO login or refresh, we >>> will be able to: >>> - ask user for re-authenticate through OTP every 30 minutes >>> - ask user for re-authenticate through OTP if he was authenticated >>> through Facebook broker >>> - ask user for re-authenticate through OTP if application required that >>> with some magic parameter (AFAIK both OIDC or SAML has some parameters >>> for specify authentication levels) >>> >>> Marek >>> >>> On 20/11/15 08:41, Marek Posolda wrote: >>>> You're right, we don't have this right now. I don't know if it's >>>> something we should support OOTB. The idea of broker login is, that >>>> you delegate authentication to another SSO/social server. Once the >>>> second server say "Ok, user is authenticated", we treat him as >>>> authenticated on Keycloak side too. Isn't it the more proper option >>>> for your usecase to use OTP on the second server side instead? >>>> >>>> Another option is to implement IdentityProviderMapper and in >>>> "updateBrokeredUser" method, you will redirect user to OTP login. >>>> Could you try this? >>>> >>>> >>>> If we want to support another login flow triggered after each broker >>>> login (which I am not convinced TBH), we can either: >>>> >>>> 1) Introduce "post-broker-login" flow, which will be configurable per >>>> IdentityProvider. By default, it will be empty . >>>> >>>> 2) Use just one flow, which will be triggered after each broker login >>>> (current "first broker login" flow is triggered only if >>>> federatedIdentity doesn't yet exist in Keycloak). In this case, the >>>> current "first broker login" flow will need to be renamed to "broker >>>> login" and more logic will need to be moved from IdentityBrokerService >>>> to the flow itself. The disadvantage of this option is, that it may >>>> always require another redirect to trigger authentication flows. But >>>> we're trying to reduce the number of redirects ( >>>> https://issues.jboss.org/browse/KEYCLOAK-2098 ) >>>> >>>> Marek >>>> >>>> On 20/11/15 00:06, Dane Barentine wrote: >>>>> Hi all, >>>>> >>>>> I?m trying to add a custom authenticator and it appears that that >>>>> there is no way to insert it in the flow if it?s a brokered IDP login >>>>> where the linked Keycloak account already exists. >>>>> >>>>> If it?s a local Keycloak user I can use the Browser flow and if it?s >>>>> a new brokered user the First Broker Login flow will execute. But I >>>>> don?t see a flow that would allow me to insert something like OTP >>>>> after a brokered login of an existing user. >>>>> >>>>> If I?m just missing it let me know but I think there needs to be some >>>>> sort of flow for brokered logins that runs on both existing and new >>>>> users. For new users it would run after the First Broker Login flow. >>>>> Or better yet maybe a flow that would allow things such as OTP to >>>>> happen after any brokered or local login. That way it wouldn?t have >>>>> to be configured in multiple flows. >>>>> >>>>> Thanks >>>>> >>>>> Dane >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From mposolda at redhat.com Fri Nov 20 11:46:33 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 20 Nov 2015 17:46:33 +0100 Subject: [keycloak-dev] no empty password in UserFederationProvider In-Reply-To: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> References: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> Message-ID: <564F4E69.9040604@redhat.com> It should be fine to remove it in Authenticator, but I am not sure if it addresses your usecase. We have another similar check in LDAPOperationManager.authenticate : https://github.com/keycloak/keycloak/blob/master/federation/ldap/src/main/java/org/keycloak/federation/ldap/idm/store/ldap/LDAPOperationManager.java#L330 . The reason behind it was another issue related to AD. Basically all other LDAP servers will always throw an exception if you put incorrect (or empty) LDAP password. However AD in some cases doesn't throw an exception in case of empty password . It throws an exception for invalid password, but not for an empty one. So it will allow any AD user to login into Keycloak with empty password... I don't know if there is some switch in AD settings to disable this behaviour. But anyway, we need to have solution, which will work in all situations. So we either need to add some switch to configuration (true: allow empty password to be sent to LDAP authentication, false: not allow it and throw error as we do now. Default setting will be false). Or if you can figure the code, which works for all usecases without additional switch, it will be even better :-) Marek On 20/11/15 16:16, Michael Gerber wrote: > AbstractUsernameFormAuthenticator.validatePassword > > public boolean validatePassword(AuthenticationFlowContext context, UserModel user, MultivaluedMap inputData) { > List credentials =new LinkedList<>(); String password = inputData.getFirst(CredentialRepresentation.PASSWORD); if (password ==null || password.isEmpty()) { > invalidPassword(context, user); return false; } > credentials.add(UserCredentialModel.password(password)); boolean valid = context.getSession().users().validCredentials(context.getRealm(), user, credentials); if (!valid) { > invalidPassword(context, user); return false; } > return true; } > I think we can remove the first if (password == null || > password.isEmpty()) > > Am 20. November 2015 um 16:11 schrieb Bill Burke : > >> Point me to the code? >> >> On 11/20/2015 9:04 AM, Michael Gerber wrote: >>> Hi All, >>> >>> keycloak does not pass an empty password to the validCredentials method >>> in the UserFederationProvider class. >>> Is there a reason for that? I would like to authenticate against an AD >>> even if the password is empty, otherwise the user won't be blocked after >>> x attempts. >>> >>> Michael >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151120/37005b82/attachment.html From mposolda at redhat.com Fri Nov 20 11:50:14 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 20 Nov 2015 17:50:14 +0100 Subject: [keycloak-dev] no empty password in UserFederationProvider In-Reply-To: <564F4D06.2070205@redhat.com> References: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> <564F4D06.2070205@redhat.com> Message-ID: <564F4F46.5090700@redhat.com> That will be the easiest path to use our BruteForceProtector. However AD also has some "BruteForceProtector" of it's own, which disables user in AD when he reach some count of invalid attempts. And I guess Michael wants to use that one and disable user in AD as well. Marek On 20/11/15 17:40, Bill Burke wrote: > You can I guess, but why does it matter? invalidPassword hits the brute > force detector if it is turned on. > > On 11/20/2015 10:16 AM, Michael Gerber wrote: >> AbstractUsernameFormAuthenticator.validatePassword >> >> public boolean validatePassword(AuthenticationFlowContext context, UserModel user, MultivaluedMap inputData) { >> List credentials =new LinkedList<>(); >> String password = inputData.getFirst(CredentialRepresentation.PASSWORD); >> if (password ==null || password.isEmpty()) { >> invalidPassword(context, user); >> return false; >> } >> credentials.add(UserCredentialModel.password(password)); >> boolean valid = context.getSession().users().validCredentials(context.getRealm(), user, credentials); >> if (!valid) { >> invalidPassword(context, user); >> return false; >> } >> return true; >> } >> >> I think we can remove the first if (password == null || password.isEmpty()) >> >> Am 20. November 2015 um 16:11 schrieb Bill Burke : >> >>> Point me to the code? >>> >>> On 11/20/2015 9:04 AM, Michael Gerber wrote: >>>> Hi All, >>>> >>>> keycloak does not pass an empty password to the validCredentials method >>>> in the UserFederationProvider class. >>>> Is there a reason for that? I would like to authenticate against an AD >>>> even if the password is empty, otherwise the user won't be blocked after >>>> x attempts. >>>> >>>> Michael >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>> -- >>> Bill Burke >>> JBoss, a division of Red Hat >>> http://bill.burkecentral.com >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev From mposolda at redhat.com Fri Nov 20 11:57:55 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 20 Nov 2015 17:57:55 +0100 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: <564F4D4A.3050805@redhat.com> References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> <564F47E4.3010809@redhat.com> <564F4D4A.3050805@redhat.com> Message-ID: <564F5113.9030202@redhat.com> Yes, however the new "First Broker Login" flow I've added is triggered just when there is no Keycloak user already linked to IdentityProvider. In case that there is already some Keycloak user linked to identityProvider identity, the flow is not executed. Should it be changed to trigger flow after each broker login? I didn't want to do that as there were no use-cases (until now) . And always trigger the flow might require another browser redirect.. Marek On 20/11/15 17:41, Bill Burke wrote: > Oh, I thought our broker flows happened after the user logged into the > external IDP? > > On 11/20/2015 11:18 AM, Marek Posolda wrote: >> Yep. I know :-\ >> >> As usually, the most proper solution requires most of the changes and >> refactoring... >> >> For this particular case, I wonder if Dane can use the approach with >> implement IdentityProviderMapper to redirect to OTP login. This is >> workaround though (IdentityProviderMapper is likely not designed for the >> usecases like this), however I think it should work. This won't require >> any changes in existing Keycloak code. >> >> I can introduce "post-broker-login" flow, but not sure if it's something >> we should support OOTB. I don't know if it's useful for more people. So >> I would rather skip adding post-broker-login as "authentication levels" >> will be more proper and more generic solution IMO. >> >> Marek >> >> >> On 20/11/15 16:05, Bill Burke wrote: >>> All something we want to do, but do we have time? Very soon we need to >>> start focusing on cleaning up the APIs/SPIs, documentation, improving >>> UI, performance, and the tons of minor problems people are >>> experiencing. >>> >>> On 11/20/2015 3:45 AM, Marek Posolda wrote: >>>> I wonder if we should address this together with reducing number of >>>> redirects ( https://issues.jboss.org/browse/KEYCLOAK-2098 ) and also >>>> support for "authentication levels" . We can encapsulate all the state >>>> of authentication in ClientSession. If someone refreshes the page, we >>>> will retrieve the ClientSession from the code and the current state of >>>> execution will be retrieved from ClientSession itself. >>>> >>>> ClientSession will also encapsulate all the state of authentication >>>> after authentication is finished. For example there will be >>>> "cookie" if >>>> ClientSession was authenticated through cookie, "password, otp" if >>>> clientSession was authenticated through password + otp or >>>> "broker-facebook" if it was authenticated through Facebook. There will >>>> be also timestamps of when each successful authentication happened. >>>> The >>>> state will be partially saved in UserSession, so if clientSession was >>>> authenticated through cookie, it can retrieve from userSession when >>>> the >>>> "real" authentication happened etc. >>>> >>>> This will allow us to support periodic re-authentication, >>>> authentication >>>> levels and different requirements for "authentication level" for >>>> individual application. For example at each SSO login or refresh, we >>>> will be able to: >>>> - ask user for re-authenticate through OTP every 30 minutes >>>> - ask user for re-authenticate through OTP if he was authenticated >>>> through Facebook broker >>>> - ask user for re-authenticate through OTP if application required >>>> that >>>> with some magic parameter (AFAIK both OIDC or SAML has some parameters >>>> for specify authentication levels) >>>> >>>> Marek >>>> >>>> On 20/11/15 08:41, Marek Posolda wrote: >>>>> You're right, we don't have this right now. I don't know if it's >>>>> something we should support OOTB. The idea of broker login is, that >>>>> you delegate authentication to another SSO/social server. Once the >>>>> second server say "Ok, user is authenticated", we treat him as >>>>> authenticated on Keycloak side too. Isn't it the more proper option >>>>> for your usecase to use OTP on the second server side instead? >>>>> >>>>> Another option is to implement IdentityProviderMapper and in >>>>> "updateBrokeredUser" method, you will redirect user to OTP login. >>>>> Could you try this? >>>>> >>>>> >>>>> If we want to support another login flow triggered after each broker >>>>> login (which I am not convinced TBH), we can either: >>>>> >>>>> 1) Introduce "post-broker-login" flow, which will be configurable per >>>>> IdentityProvider. By default, it will be empty . >>>>> >>>>> 2) Use just one flow, which will be triggered after each broker login >>>>> (current "first broker login" flow is triggered only if >>>>> federatedIdentity doesn't yet exist in Keycloak). In this case, the >>>>> current "first broker login" flow will need to be renamed to "broker >>>>> login" and more logic will need to be moved from >>>>> IdentityBrokerService >>>>> to the flow itself. The disadvantage of this option is, that it may >>>>> always require another redirect to trigger authentication flows. But >>>>> we're trying to reduce the number of redirects ( >>>>> https://issues.jboss.org/browse/KEYCLOAK-2098 ) >>>>> >>>>> Marek >>>>> >>>>> On 20/11/15 00:06, Dane Barentine wrote: >>>>>> Hi all, >>>>>> >>>>>> I?m trying to add a custom authenticator and it appears that that >>>>>> there is no way to insert it in the flow if it?s a brokered IDP >>>>>> login >>>>>> where the linked Keycloak account already exists. >>>>>> >>>>>> If it?s a local Keycloak user I can use the Browser flow and if it?s >>>>>> a new brokered user the First Broker Login flow will execute. But I >>>>>> don?t see a flow that would allow me to insert something like OTP >>>>>> after a brokered login of an existing user. >>>>>> >>>>>> If I?m just missing it let me know but I think there needs to be >>>>>> some >>>>>> sort of flow for brokered logins that runs on both existing and new >>>>>> users. For new users it would run after the First Broker Login flow. >>>>>> Or better yet maybe a flow that would allow things such as OTP to >>>>>> happen after any brokered or local login. That way it wouldn?t have >>>>>> to be configured in multiple flows. >>>>>> >>>>>> Thanks >>>>>> >>>>>> Dane >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> > From bburke at redhat.com Fri Nov 20 12:22:50 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 20 Nov 2015 12:22:50 -0500 Subject: [keycloak-dev] groups in Message-ID: <564F56EA.3080908@redhat.com> * Groups are hierarchical. One parent, multiple children. UI is a tree. Probably needs some UIX love. I couldn't get a drag-drop Tree widget working...As a result, you have to use a combination of button presses to add a group to a parent. (new/cut/paste) * Attribute and Role mappers will now take group membership into account when mapping tokens and SAML assertions. * Default Groups added. Same as Default Roles, accept for groups. * There are new GroupMappers added to SAML and OIDC mappers. They are not on by default. This work probably should have been done a week or two ago, but oh well...I'm inefficient lately... -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From gerbermichi at me.com Fri Nov 20 12:22:53 2015 From: gerbermichi at me.com (Michael Gerber) Date: Fri, 20 Nov 2015 18:22:53 +0100 Subject: [keycloak-dev] no empty password in UserFederationProvider In-Reply-To: <564F4F46.5090700@redhat.com> References: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> <564F4D06.2070205@redhat.com> <564F4F46.5090700@redhat.com> Message-ID: We?ve got a custom UserFederationProvider, which authenticate users against an AD or DB. Therefore, we need to know if a user entered an empty password. I will create a PR and jira ticket for that, ok? > On 20.11.2015, at 17:50, Marek Posolda wrote: > > That will be the easiest path to use our BruteForceProtector. > > However AD also has some "BruteForceProtector" of it's own, which disables user in AD when he reach some count of invalid attempts. And I guess Michael wants to use that one and disable user in AD as well. > > Marek > > On 20/11/15 17:40, Bill Burke wrote: >> You can I guess, but why does it matter? invalidPassword hits the brute >> force detector if it is turned on. >> >> On 11/20/2015 10:16 AM, Michael Gerber wrote: >>> AbstractUsernameFormAuthenticator.validatePassword >>> >>> public boolean validatePassword(AuthenticationFlowContext context, UserModel user, MultivaluedMap inputData) { >>> List credentials =new LinkedList<>(); >>> String password = inputData.getFirst(CredentialRepresentation.PASSWORD); >>> if (password ==null || password.isEmpty()) { >>> invalidPassword(context, user); >>> return false; >>> } >>> credentials.add(UserCredentialModel.password(password)); >>> boolean valid = context.getSession().users().validCredentials(context.getRealm(), user, credentials); >>> if (!valid) { >>> invalidPassword(context, user); >>> return false; >>> } >>> return true; >>> } >>> >>> I think we can remove the first if (password == null || password.isEmpty()) >>> >>> Am 20. November 2015 um 16:11 schrieb Bill Burke : >>> >>>> Point me to the code? >>>> >>>> On 11/20/2015 9:04 AM, Michael Gerber wrote: >>>>> Hi All, >>>>> >>>>> keycloak does not pass an empty password to the validCredentials method >>>>> in the UserFederationProvider class. >>>>> Is there a reason for that? I would like to authenticate against an AD >>>>> even if the password is empty, otherwise the user won't be blocked after >>>>> x attempts. >>>>> >>>>> Michael >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>> -- >>>> Bill Burke >>>> JBoss, a division of Red Hat >>>> http://bill.burkecentral.com >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > From Dane.Barentine at software.dell.com Fri Nov 20 12:46:39 2015 From: Dane.Barentine at software.dell.com (Dane Barentine) Date: Fri, 20 Nov 2015 17:46:39 +0000 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: <564F5113.9030202@redhat.com> References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> <564F47E4.3010809@redhat.com> <564F4D4A.3050805@redhat.com> <564F5113.9030202@redhat.com> Message-ID: I can look at the IdentityProvider Mapper and see. As you said it's a workaround though and now requires me to create a mapper and an authenticator to handle both types of authentication. Regarding this: "Isn't it the more proper option for your usecase to use OTP on the second server side instead?" In a lot of cases it would probably be more proper. But it's not always practical as we don't always control the IDPs. So the use case for OTP is if we are using an IDP that either doesn't support, or the team that is running it doesn't want to support, something like OTP but we still want to require it when logging in through Keycloak. The other use case is when we are making authentication/authorization decisions based on some other sort risk assessment or user required action. This may involve using services or code that we will never be able to plug into an IDP. In those cases a post broker login flow would allow us to add on a consistent layer across all the brokered IDPs regardless of what they can technically support. Dane -----Original Message----- From: keycloak-dev-bounces at lists.jboss.org [mailto:keycloak-dev-bounces at lists.jboss.org] On Behalf Of Marek Posolda Sent: Friday, November 20, 2015 8:58 AM To: Bill Burke ; keycloak-dev at lists.jboss.org Subject: Re: [keycloak-dev] Identity Broker login flow Yes, however the new "First Broker Login" flow I've added is triggered just when there is no Keycloak user already linked to IdentityProvider. In case that there is already some Keycloak user linked to identityProvider identity, the flow is not executed. Should it be changed to trigger flow after each broker login? I didn't want to do that as there were no use-cases (until now) . And always trigger the flow might require another browser redirect.. Marek On 20/11/15 17:41, Bill Burke wrote: > Oh, I thought our broker flows happened after the user logged into the > external IDP? > > On 11/20/2015 11:18 AM, Marek Posolda wrote: >> Yep. I know :-\ >> >> As usually, the most proper solution requires most of the changes and >> refactoring... >> >> For this particular case, I wonder if Dane can use the approach with >> implement IdentityProviderMapper to redirect to OTP login. This is >> workaround though (IdentityProviderMapper is likely not designed for >> the usecases like this), however I think it should work. This won't >> require any changes in existing Keycloak code. >> >> I can introduce "post-broker-login" flow, but not sure if it's >> something we should support OOTB. I don't know if it's useful for >> more people. So I would rather skip adding post-broker-login as "authentication levels" >> will be more proper and more generic solution IMO. >> >> Marek >> >> >> On 20/11/15 16:05, Bill Burke wrote: >>> All something we want to do, but do we have time? Very soon we need >>> to start focusing on cleaning up the APIs/SPIs, documentation, >>> improving UI, performance, and the tons of minor problems people are >>> experiencing. >>> >>> On 11/20/2015 3:45 AM, Marek Posolda wrote: >>>> I wonder if we should address this together with reducing number of >>>> redirects ( https://issues.jboss.org/browse/KEYCLOAK-2098 ) and >>>> also support for "authentication levels" . We can encapsulate all >>>> the state of authentication in ClientSession. If someone refreshes >>>> the page, we will retrieve the ClientSession from the code and the >>>> current state of execution will be retrieved from ClientSession itself. >>>> >>>> ClientSession will also encapsulate all the state of authentication >>>> after authentication is finished. For example there will be >>>> "cookie" if ClientSession was authenticated through cookie, >>>> "password, otp" if clientSession was authenticated through password >>>> + otp or "broker-facebook" if it was authenticated through >>>> Facebook. There will be also timestamps of when each successful >>>> authentication happened. >>>> The >>>> state will be partially saved in UserSession, so if clientSession >>>> was authenticated through cookie, it can retrieve from userSession >>>> when the "real" authentication happened etc. >>>> >>>> This will allow us to support periodic re-authentication, >>>> authentication levels and different requirements for >>>> "authentication level" for individual application. For example at >>>> each SSO login or refresh, we will be able to: >>>> - ask user for re-authenticate through OTP every 30 minutes >>>> - ask user for re-authenticate through OTP if he was authenticated >>>> through Facebook broker >>>> - ask user for re-authenticate through OTP if application required >>>> that with some magic parameter (AFAIK both OIDC or SAML has some >>>> parameters for specify authentication levels) >>>> >>>> Marek >>>> >>>> On 20/11/15 08:41, Marek Posolda wrote: >>>>> You're right, we don't have this right now. I don't know if it's >>>>> something we should support OOTB. The idea of broker login is, >>>>> that you delegate authentication to another SSO/social server. >>>>> Once the second server say "Ok, user is authenticated", we treat >>>>> him as authenticated on Keycloak side too. Isn't it the more >>>>> proper option for your usecase to use OTP on the second server side instead? >>>>> >>>>> Another option is to implement IdentityProviderMapper and in >>>>> "updateBrokeredUser" method, you will redirect user to OTP login. >>>>> Could you try this? >>>>> >>>>> >>>>> If we want to support another login flow triggered after each >>>>> broker login (which I am not convinced TBH), we can either: >>>>> >>>>> 1) Introduce "post-broker-login" flow, which will be configurable >>>>> per IdentityProvider. By default, it will be empty . >>>>> >>>>> 2) Use just one flow, which will be triggered after each broker >>>>> login (current "first broker login" flow is triggered only if >>>>> federatedIdentity doesn't yet exist in Keycloak). In this case, >>>>> the current "first broker login" flow will need to be renamed to >>>>> "broker login" and more logic will need to be moved from >>>>> IdentityBrokerService to the flow itself. The disadvantage of this >>>>> option is, that it may always require another redirect to trigger >>>>> authentication flows. But we're trying to reduce the number of >>>>> redirects ( >>>>> https://issues.jboss.org/browse/KEYCLOAK-2098 ) >>>>> >>>>> Marek >>>>> >>>>> On 20/11/15 00:06, Dane Barentine wrote: >>>>>> Hi all, >>>>>> >>>>>> I'm trying to add a custom authenticator and it appears that that >>>>>> there is no way to insert it in the flow if it's a brokered IDP >>>>>> login where the linked Keycloak account already exists. >>>>>> >>>>>> If it's a local Keycloak user I can use the Browser flow and if >>>>>> it's a new brokered user the First Broker Login flow will >>>>>> execute. But I don't see a flow that would allow me to insert >>>>>> something like OTP after a brokered login of an existing user. >>>>>> >>>>>> If I'm just missing it let me know but I think there needs to be >>>>>> some sort of flow for brokered logins that runs on both existing >>>>>> and new users. For new users it would run after the First Broker >>>>>> Login flow. >>>>>> Or better yet maybe a flow that would allow things such as OTP to >>>>>> happen after any brokered or local login. That way it wouldn't >>>>>> have to be configured in multiple flows. >>>>>> >>>>>> Thanks >>>>>> >>>>>> Dane >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >> > _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From ayoung at redhat.com Sun Nov 22 21:13:17 2015 From: ayoung at redhat.com (Adam Young) Date: Sun, 22 Nov 2015 21:13:17 -0500 Subject: [keycloak-dev] user groups vs. client groups Message-ID: <5652763D.3050304@redhat.com> Hey guys, just catching up on the discussion. Let me share some lessons learned from the Last four years on OpenStack Keystone working on a very similar problem. Keystone started off with three concepts: users, (global) roles and tenants. A user was either an admin (based on a role) or assigned to a tenant. Tenants were groups of resources not managed by Keystone; Virtual machines, disk images, and the like. All access control was managed by the user having a token scoped to the project. Admin was a global role...but at some point, someone decided to make it scoped to a tenant. And things go weird. But it made sense: within a proejct, some resources were more sensitive than others, and needed an increased level of security. But since all of the access control policy was written with global roles, things were actually just kind of broken. It lead to one of the oldest bugs in Keystone, and it is still assigned to me. I think we will have it licked this round...but that is jumping ahead. Keystone was written with a plugable backend approach. THe default was Key Value stores (memcache was heavily used) but SQL was very dominant. However, we realized that, for users, we needed to be able to pull them out of LDAP, as most companies have large, embedded user databases, and they want to user them, not copy them. So, projects in Openstack mapped to groups in LDAP, and users had assignments to groups. It was an uncomfortable fit. Tenants really were OpenStack abstractions, and with them coming from LDAP...which was almost always read-only, they were not editable by OpenStack resources. This was a restriction that we dealt with for a while. A couple new abstractions found their way into Keystone: groups (of users) and domains. Domains were groupings for both users and tenants. And this has continued to haunt us, because it confused two things. The above situation with LDAP showed that the users needed to be separate from the tenants. Oh, and we also changed the name from tenants to projects to keep consistent with the rest of OpenStack...want to piss off your users? Change a key term out from underneath them... It became clear the we needed to manage users and projects in different ways. Users were read only from LDAP, projects were writable from OpenStack. We split the backends along these lines: Identity manages users, groups, and user-to-group assignments. This was due to the4 expectation that the group information from LDAP was going to be required for managing users, which has proven to be of questionable worth. The assignment backend held projects, roles, and role assignments. Domains were in there as well, but it was an awkward fit, as domains really spanned both Identity and Assignment. We also changed how the user to projects mapping was handled: we use role assignments for everything. Domain *owned* users, in that they were records under the domain object, but a user got no permissions due to being managed by a specific domain; only via role assignments. Things really got funky when we fully integrated Federation in to Keystone. Now, we had two abstractions for managing users: Domains and Identity Providers. Instead of making a hard relationship between the two...we sort of let them float. The most hackish part is the Mapping. This is what takes the values from the federation assertion (SAML for the most part) and maps it to Keystone Identity objects (users and groups). While we should limit what IdP can map to what domain, we don't. What this means is the managing the mapping must be done by an admin, and that is a cetrnalization I had hoped to avoid. What do end users really need? They need a reusable set of role assignments that span mulitple (micro)services. We are almost there, and a few more iterations and we can get there. Keystone handles only the core OpenStack services, but it would be much more useful if we could open it up to the cloud in general, and make it so people could use the Federation mapping defined in there to provide Access control to applications running in the cloud. Lessons learned: Scope roles to your target abstraction from the beginning. Call this Scoped RBAC, as opposed to what NIST does. NIST is implicitly scoped, make KeyCLoack be explicit. Call it project, tenant, target, application, I don't care; but a user should have a role on *something*, otherwise you end up with namespacing inside the role names, and a second level of mapping required. Provide rules that allow one to explicitly assign a single role, and the target user gets that explicit role and multiple implicit roles. The end product is actually as set, not a tree, so just watch out for cycles. It does not have to be a strict hierarchy, so long as it is a Directed Acyclic Graph, you are safe. Namespacing of roles is a client organization thing; it does not need to end up in the roles shipped to the end application, so long as you have a global definition of what roles mean what. Regardless, the namespace needs to disappear when sending the assertion, or the remote side can;t use it. So, knowing "who" you are getting an assertion for is essential. Two different Wordpress installs mean two different role assignments: admin on one, editor on the other. You don't own the user database. Assume it is going to be read only. Provide a local means to manage groups of users separate from, but on top of, the groups that come from the federated source of Identity. OK, that is a lot of a brain dump from someone you all don't even know. PLease ask me questions about the details. I'm pretty impressed with what I've seen of KeyCloak thus far. Looking forward to getting to know it better. From bburke at redhat.com Sun Nov 22 22:50:41 2015 From: bburke at redhat.com (Bill Burke) Date: Sun, 22 Nov 2015 22:50:41 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <5652763D.3050304@redhat.com> References: <5652763D.3050304@redhat.com> Message-ID: <56528D11.3010308@redhat.com> We should have a Google Hangout or BlueJeans sometime: before I get deeper, you'll have to explain more on what you mean that you wanted to avoid centralization of mapping SAML assertions. I didn't get that part... I think this is related to a lot of what you wrote, but resource permissions are going to be a separate, but integrated service to roles/users/groups. Pedro is working on a permission service that allows you to have complex permission models. There's some nice protocols out there that help manage this. Most Java EE apps, simple permission stuff is handled by the applications themselves. The security layer propagates identity and user/role mappings and the application decides which roles have permission to access which resource. So, Keycloak core can focus on provide user/group/role/attribute authentication and information. The service Pedro is working on will offer an optional, complimentary way to centrally manage permissions if you want to. My on personal opinion is that most apps will want to manage resource permissions in their own special app-specific way and Pedro's stuff will be most useful as a backend-service. As for our architecture, we have a common security domain model that all our core component work with. This is a pluggable abstraction that is designed to sit on a pluggable datastore. We currently have SQL and Mongo mappings sitting on top of our own datamodel. We have a distributed, clustered cache sitting on top of our datamodel (Infinispan). It uses an invalidation model to keep synching message size minimal and to avoid transmiting sensitive data. For federation of external identity stores (i.e. LDAP) we offer a hybrid approach. The idea is that we want Keycloaok to have a set of features that a user can leverage irregardless of whether the underlying external store can handle it or not. This means that Keylocak might locally store additional security metadata that is not available in the LDAP store. The SPI is a bit complex right now, but we're hoping to fix that. It also requires some minimal import of data for each user (username/email). We want to code that requirement out though in the future. On 11/22/2015 9:13 PM, Adam Young wrote: > Hey guys, just catching up on the discussion. Let me share some > lessons learned from the Last four years on OpenStack Keystone working > on a very similar problem. > > Keystone started off with three concepts: users, (global) roles and > tenants. A user was either an admin (based on a role) or assigned to a > tenant. > > Tenants were groups of resources not managed by Keystone; Virtual > machines, disk images, and the like. All access control was managed by > the user having a token scoped to the project. > > Admin was a global role...but at some point, someone decided to make it > scoped to a tenant. And things go weird. But it made sense: within a > proejct, some resources were more sensitive than others, and needed an > increased level of security. But since all of the access control policy > was written with global roles, things were actually just kind of > broken. It lead to one of the oldest bugs in Keystone, and it is still > assigned to me. I think we will have it licked this round...but that is > jumping ahead. > > Keystone was written with a plugable backend approach. THe default was > Key Value stores (memcache was heavily used) but SQL was very dominant. > However, we realized that, for users, we needed to be able to pull them > out of LDAP, as most companies have large, embedded user databases, and > they want to user them, not copy them. So, projects in Openstack mapped > to groups in LDAP, and users had assignments to groups. > > It was an uncomfortable fit. Tenants really were OpenStack > abstractions, and with them coming from LDAP...which was almost always > read-only, they were not editable by OpenStack resources. This was a > restriction that we dealt with for a while. > > A couple new abstractions found their way into Keystone: groups (of > users) and domains. Domains were groupings for both users and tenants. > And this has continued to haunt us, because it confused two things. The > above situation with LDAP showed that the users needed to be separate > from the tenants. > > Oh, and we also changed the name from tenants to projects to keep > consistent with the rest of OpenStack...want to piss off your users? > Change a key term out from underneath them... > > > It became clear the we needed to manage users and projects in different > ways. Users were read only from LDAP, projects were writable from > OpenStack. We split the backends along these lines: Identity manages > users, groups, and user-to-group assignments. This was due to the4 > expectation that the group information from LDAP was going to be > required for managing users, which has proven to be of questionable > worth. The assignment backend held projects, roles, and role > assignments. Domains were in there as well, but it was an awkward fit, > as domains really spanned both Identity and Assignment. > > > We also changed how the user to projects mapping was handled: we use > role assignments for everything. Domain *owned* users, in that they > were records under the domain object, but a user got no permissions due > to being managed by a specific domain; only via role assignments. > > Things really got funky when we fully integrated Federation in to > Keystone. Now, we had two abstractions for managing users: Domains and > Identity Providers. Instead of making a hard relationship between the > two...we sort of let them float. > > The most hackish part is the Mapping. This is what takes the values from > the federation assertion (SAML for the most part) and maps it to > Keystone Identity objects (users and groups). While we should limit > what IdP can map to what domain, we don't. What this means is the > managing the mapping must be done by an admin, and that is a > cetrnalization I had hoped to avoid. > > > What do end users really need? They need a reusable set of role > assignments that span mulitple (micro)services. We are almost there, > and a few more iterations and we can get there. Keystone handles only > the core OpenStack services, but it would be much more useful if we > could open it up to the cloud in general, and make it so people could > use the Federation mapping defined in there to provide Access control to > applications running in the cloud. > > > Lessons learned: > > Scope roles to your target abstraction from the beginning. Call this > Scoped RBAC, as opposed to what NIST does. NIST is implicitly scoped, > make KeyCLoack be explicit. Call it project, tenant, target, > application, I don't care; but a user should have a role on > *something*, otherwise you end up with namespacing inside the role > names, and a second level of mapping required. > > > Provide rules that allow one to explicitly assign a single role, and the > target user gets that explicit role and multiple implicit roles. The > end product is actually as set, not a tree, so just watch out for > cycles. It does not have to be a strict hierarchy, so long as it is a > Directed Acyclic Graph, you are safe. > > Namespacing of roles is a client organization thing; it does not need to > end up in the roles shipped to the end application, so long as you have > a global definition of what roles mean what. Regardless, the namespace > needs to disappear when sending the assertion, or the remote side can;t > use it. So, knowing "who" you are getting an assertion for is > essential. Two different Wordpress installs mean two different role > assignments: admin on one, editor on the other. > > > You don't own the user database. Assume it is going to be read only. > Provide a local means to manage groups of users separate from, but on > top of, the groups that come from the federated source of Identity. > > > > OK, that is a lot of a brain dump from someone you all don't even know. > PLease ask me questions about the details. > > > I'm pretty impressed with what I've seen of KeyCloak thus far. Looking > forward to getting to know it better. > > > > > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From mposolda at redhat.com Mon Nov 23 02:54:37 2015 From: mposolda at redhat.com (Marek Posolda) Date: Mon, 23 Nov 2015 08:54:37 +0100 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> <564F47E4.3010809@redhat.com> <564F4D4A.3050805@redhat.com> <564F5113.9030202@redhat.com> Message-ID: <5652C63D.7070106@redhat.com> On 20/11/15 18:46, Dane Barentine wrote: > I can look at the IdentityProvider Mapper and see. As you said it's a workaround though and now requires me to create a mapper and an authenticator to handle both types of authentication. > > Regarding this: "Isn't it the more proper option for your usecase to use OTP on the second server side instead?" > > In a lot of cases it would probably be more proper. But it's not always practical as we don't always control the IDPs. So the use case for OTP is if we are using an IDP that either doesn't support, or the team that is running it doesn't want to support, something like OTP but we still want to require it when logging in through Keycloak. The other use case is when we are making authentication/authorization decisions based on some other sort risk assessment or user required action. This may involve using services or code that we will never be able to plug into an IDP. In those cases a post broker login flow would allow us to add on a consistent layer across all the brokered IDPs regardless of what they can technically support. You can use requiredAction SPI after broker authentication. That is triggered after each authentication (classic or broker) . See some docs here : http://keycloak.github.io/docs/userguide/keycloak-server/html/auth_spi.html#d4e3420 . Maybe using RequiredAction SPI for OTP authentication is even better than IdentityProviderMapper (even if it's still seems to be a workaround). In your RequiredActionProvider.evaluateTriggers(), you will check that user was authenticated through broker and if yes, you put some requiredAction like "authenticate_otp" to clientSession. Then in requiredActionChallenge() you will redirect to OTP form and in processAction() you will check if OTP was successful. The code in requiredActionChallenge and processAction methods might be very similar to the code in OTPFormAuthenticator. You can likely inspire from here. If you still have issues, feel free to create JIRA and we will try to look at improve things in Keycloak. Regards, Marek > > Dane > > -----Original Message----- > From: keycloak-dev-bounces at lists.jboss.org [mailto:keycloak-dev-bounces at lists.jboss.org] On Behalf Of Marek Posolda > Sent: Friday, November 20, 2015 8:58 AM > To: Bill Burke ; keycloak-dev at lists.jboss.org > Subject: Re: [keycloak-dev] Identity Broker login flow > > Yes, however the new "First Broker Login" flow I've added is triggered just when there is no Keycloak user already linked to IdentityProvider. > In case that there is already some Keycloak user linked to identityProvider identity, the flow is not executed. Should it be changed to trigger flow after each broker login? I didn't want to do that as there were no use-cases (until now) . And always trigger the flow might require another browser redirect.. > > Marek > > On 20/11/15 17:41, Bill Burke wrote: >> Oh, I thought our broker flows happened after the user logged into the >> external IDP? >> >> On 11/20/2015 11:18 AM, Marek Posolda wrote: >>> Yep. I know :-\ >>> >>> As usually, the most proper solution requires most of the changes and >>> refactoring... >>> >>> For this particular case, I wonder if Dane can use the approach with >>> implement IdentityProviderMapper to redirect to OTP login. This is >>> workaround though (IdentityProviderMapper is likely not designed for >>> the usecases like this), however I think it should work. This won't >>> require any changes in existing Keycloak code. >>> >>> I can introduce "post-broker-login" flow, but not sure if it's >>> something we should support OOTB. I don't know if it's useful for >>> more people. So I would rather skip adding post-broker-login as "authentication levels" >>> will be more proper and more generic solution IMO. >>> >>> Marek >>> >>> >>> On 20/11/15 16:05, Bill Burke wrote: >>>> All something we want to do, but do we have time? Very soon we need >>>> to start focusing on cleaning up the APIs/SPIs, documentation, >>>> improving UI, performance, and the tons of minor problems people are >>>> experiencing. >>>> >>>> On 11/20/2015 3:45 AM, Marek Posolda wrote: >>>>> I wonder if we should address this together with reducing number of >>>>> redirects ( https://issues.jboss.org/browse/KEYCLOAK-2098 ) and >>>>> also support for "authentication levels" . We can encapsulate all >>>>> the state of authentication in ClientSession. If someone refreshes >>>>> the page, we will retrieve the ClientSession from the code and the >>>>> current state of execution will be retrieved from ClientSession itself. >>>>> >>>>> ClientSession will also encapsulate all the state of authentication >>>>> after authentication is finished. For example there will be >>>>> "cookie" if ClientSession was authenticated through cookie, >>>>> "password, otp" if clientSession was authenticated through password >>>>> + otp or "broker-facebook" if it was authenticated through >>>>> Facebook. There will be also timestamps of when each successful >>>>> authentication happened. >>>>> The >>>>> state will be partially saved in UserSession, so if clientSession >>>>> was authenticated through cookie, it can retrieve from userSession >>>>> when the "real" authentication happened etc. >>>>> >>>>> This will allow us to support periodic re-authentication, >>>>> authentication levels and different requirements for >>>>> "authentication level" for individual application. For example at >>>>> each SSO login or refresh, we will be able to: >>>>> - ask user for re-authenticate through OTP every 30 minutes >>>>> - ask user for re-authenticate through OTP if he was authenticated >>>>> through Facebook broker >>>>> - ask user for re-authenticate through OTP if application required >>>>> that with some magic parameter (AFAIK both OIDC or SAML has some >>>>> parameters for specify authentication levels) >>>>> >>>>> Marek >>>>> >>>>> On 20/11/15 08:41, Marek Posolda wrote: >>>>>> You're right, we don't have this right now. I don't know if it's >>>>>> something we should support OOTB. The idea of broker login is, >>>>>> that you delegate authentication to another SSO/social server. >>>>>> Once the second server say "Ok, user is authenticated", we treat >>>>>> him as authenticated on Keycloak side too. Isn't it the more >>>>>> proper option for your usecase to use OTP on the second server side instead? >>>>>> >>>>>> Another option is to implement IdentityProviderMapper and in >>>>>> "updateBrokeredUser" method, you will redirect user to OTP login. >>>>>> Could you try this? >>>>>> >>>>>> >>>>>> If we want to support another login flow triggered after each >>>>>> broker login (which I am not convinced TBH), we can either: >>>>>> >>>>>> 1) Introduce "post-broker-login" flow, which will be configurable >>>>>> per IdentityProvider. By default, it will be empty . >>>>>> >>>>>> 2) Use just one flow, which will be triggered after each broker >>>>>> login (current "first broker login" flow is triggered only if >>>>>> federatedIdentity doesn't yet exist in Keycloak). In this case, >>>>>> the current "first broker login" flow will need to be renamed to >>>>>> "broker login" and more logic will need to be moved from >>>>>> IdentityBrokerService to the flow itself. The disadvantage of this >>>>>> option is, that it may always require another redirect to trigger >>>>>> authentication flows. But we're trying to reduce the number of >>>>>> redirects ( >>>>>> https://issues.jboss.org/browse/KEYCLOAK-2098 ) >>>>>> >>>>>> Marek >>>>>> >>>>>> On 20/11/15 00:06, Dane Barentine wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> I'm trying to add a custom authenticator and it appears that that >>>>>>> there is no way to insert it in the flow if it's a brokered IDP >>>>>>> login where the linked Keycloak account already exists. >>>>>>> >>>>>>> If it's a local Keycloak user I can use the Browser flow and if >>>>>>> it's a new brokered user the First Broker Login flow will >>>>>>> execute. But I don't see a flow that would allow me to insert >>>>>>> something like OTP after a brokered login of an existing user. >>>>>>> >>>>>>> If I'm just missing it let me know but I think there needs to be >>>>>>> some sort of flow for brokered logins that runs on both existing >>>>>>> and new users. For new users it would run after the First Broker >>>>>>> Login flow. >>>>>>> Or better yet maybe a flow that would allow things such as OTP to >>>>>>> happen after any brokered or local login. That way it wouldn't >>>>>>> have to be configured in multiple flows. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Dane >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> keycloak-dev mailing list >>>>>>> keycloak-dev at lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/1c1eba23/attachment-0001.html From mposolda at redhat.com Mon Nov 23 03:02:14 2015 From: mposolda at redhat.com (Marek Posolda) Date: Mon, 23 Nov 2015 09:02:14 +0100 Subject: [keycloak-dev] no empty password in UserFederationProvider In-Reply-To: References: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> <564F4D06.2070205@redhat.com> <564F4F46.5090700@redhat.com> Message-ID: <5652C806.4090502@redhat.com> Sure, that would be fine. But again, it's important that with default settings, LDAPOperationManager.authenticate won't automatically authenticate AD users with empty password (which may happen when anonymous bind is enabled on AD side). Marek On 20/11/15 18:22, Michael Gerber wrote: > We?ve got a custom UserFederationProvider, which authenticate users against an AD or DB. Therefore, we need to know if a user entered an empty password. > I will create a PR and jira ticket for that, ok? > >> On 20.11.2015, at 17:50, Marek Posolda wrote: >> >> That will be the easiest path to use our BruteForceProtector. >> >> However AD also has some "BruteForceProtector" of it's own, which disables user in AD when he reach some count of invalid attempts. And I guess Michael wants to use that one and disable user in AD as well. >> >> Marek >> >> On 20/11/15 17:40, Bill Burke wrote: >>> You can I guess, but why does it matter? invalidPassword hits the brute >>> force detector if it is turned on. >>> >>> On 11/20/2015 10:16 AM, Michael Gerber wrote: >>>> AbstractUsernameFormAuthenticator.validatePassword >>>> >>>> public boolean validatePassword(AuthenticationFlowContext context, UserModel user, MultivaluedMap inputData) { >>>> List credentials =new LinkedList<>(); >>>> String password = inputData.getFirst(CredentialRepresentation.PASSWORD); >>>> if (password ==null || password.isEmpty()) { >>>> invalidPassword(context, user); >>>> return false; >>>> } >>>> credentials.add(UserCredentialModel.password(password)); >>>> boolean valid = context.getSession().users().validCredentials(context.getRealm(), user, credentials); >>>> if (!valid) { >>>> invalidPassword(context, user); >>>> return false; >>>> } >>>> return true; >>>> } >>>> >>>> I think we can remove the first if (password == null || password.isEmpty()) >>>> >>>> Am 20. November 2015 um 16:11 schrieb Bill Burke : >>>> >>>>> Point me to the code? >>>>> >>>>> On 11/20/2015 9:04 AM, Michael Gerber wrote: >>>>>> Hi All, >>>>>> >>>>>> keycloak does not pass an empty password to the validCredentials method >>>>>> in the UserFederationProvider class. >>>>>> Is there a reason for that? I would like to authenticate against an AD >>>>>> even if the password is empty, otherwise the user won't be blocked after >>>>>> x attempts. >>>>>> >>>>>> Michael >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >>>>> -- >>>>> Bill Burke >>>>> JBoss, a division of Red Hat >>>>> http://bill.burkecentral.com >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev From sthorger at redhat.com Mon Nov 23 03:13:21 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 09:13:21 +0100 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: <564DF0AE.5050809@redhat.com> References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> <564DF0AE.5050809@redhat.com> Message-ID: On 19 November 2015 at 16:54, Marek Posolda wrote: > On 19/11/15 14:59, Stian Thorgersen wrote: > > How about: > > * IDs are usually generated, but can be specified. It should not be > possible to change them though > > * Instead of alias, name and all other ways of setting a human readable > name we add Display Name and Display Class > > So this will be for all other objects, not just clients? Again +1 as > currently it's mess. Various objects have various "names" for kind of same > thing. Or even more they have same names, for different things. For example: > - IdentityProviderModel has: internalId + alias > - UserFederationModel has: id + displayName > - role has: id + name + description (Name is unchangeable and not > localized, so have different meaning then "name" for client etc) > > But not sure what is "Display Class" . Is it the same thing like > "description" ? I would personally use: > - ID > - displayName > - description > Display class is to set a custom css class. For example someone may want to display a picture logo for a realm instead of text. > > > Not sure how to deal with migration, but if we agree on those points, then > we can figure that out > > > On 19 November 2015 at 14:49, Bill Burke wrote: > >> One more thing. Another problem is that if you switch to ID for >> everything, then demos etc. will no longer run out of the box and you'll >> have to modify adapter configuration to set the newly generated DB id. >> Really, any piece of configuration that references things by client ID >> will no longer work. >> >> So, there's really 2 issues: >> >> * Using ClientID in admin-console URLs doesn't work well as the client >> IDs could have '/' characters in them >> * Using ID only messes up pre-defined, pre-packaged adapter config and >> you have to edit these files after the keycloak DB is set up. >> >> On 11/19/2015 8:40 AM, Bill Burke wrote: >> > We currently actually have 3 client identifers: ID, ClientID, and Name. >> > >> > Also, I think there are a lot of duplicate clientID names between >> > realms. i.e. "admin-console" "broker", etc... >> > >> > A search by clientID is (realm + clientID). >> > >> > Marek is right about why I switched everything to ID in the admin >> > console. For SAML I just didn't want to add yet another client >> > identifier since we already had 3. >> > >> > On 11/19/2015 7:52 AM, Marek Posolda wrote: >> >> +1 for this change. >> >> >> >> I am just not sure if we should set the "id" to the current value of >> >> "client-id" ? Few things to note: >> >> >> >> - SAML clients currently use clientId in form of URL. For example in >> >> SAML demo there are clientIds like " >> http://localhosT:8080/employee-sig" >> >> . I don't know if it's requirement, maybe it's possible to solve it >> >> somehow (ie. introduce different attribute for SAML client to store >> >> these URLs). But from what I remember, Bill changed admin console to >> use >> >> "id" instead of "clientId" because there were issues with URL-likeadmin >> >> >> clientId in admin console . So if we overwrite the "id" with current >> >> "client-id" the issue will be back. >> >> >> >> - Migration might be a pain. Many tables (roles, protocolMappers, user >> >> consents, offline clientSessions ...) references client by "id" . >> >> Overwriting "id" with "client-id" means that we will need to change all >> >> those DB records. And there are things like foreign keys etc... >> >> >> >> Shouldn't do vice-versa and just remove current "client-id" and ask >> >> people to update their keycloak.json adapter configurations? On the >> >> other hand, removing "client-id" might break migration of JSON exported >> >> realms as the JSON entities are using "client-id" for referencing >> client. >> >> >> >> It seems the migration will be a pain regardless of whatever direction >> >> we choose :-( >> >> >> >> Marek >> >> >> >> On 16/11/15 14:54, Stian Thorgersen wrote: >> >>> We have both "id" and "client-id" for clients in Keycloak at the >> >>> moment. This seems unnecessary and complex. >> >>> >> >>> The model can retrieve clients on either value. In token endpoints the >> >>> "client-id" is used. In admin endpoints the "id" is used. >> >>> >> >>> Also, in most cases it would be simpler for users to just have a >> >>> generated id than having to come up with one themselves. The id >> >>> doesn't have to be human readable either as we have name for that. >> >>> >> >>> OpenID Connect expects "client-id" to be generated by the IdP and >> >>> can't be changed once created. >> >>> >> >>> I propose we remove "client-id" and only keep id. >> >>> >> >>> For migration of existing clients we would set the "id" value to the >> >>> current value of "client-id". This would require no changes to adapter >> >>> configs. When creating new clients from the admin console we would not >> >>> allow setting the "client-id", instead just display it after the >> >>> client was created. When importing clients it would be possible to set >> >>> the id (and for backwards compatibility we would set "id" equal to the >> >>> "client-id" field. >> >>> >> >>> >> >>> _______________________________________________ >> >>> keycloak-dev mailing list >> >>> keycloak-dev at lists.jboss.org >> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> >> >> >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/c590dc30/attachment.html From sthorger at redhat.com Mon Nov 23 03:19:32 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 09:19:32 +0100 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: <564E1441.4000604@redhat.com> References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> <564DDFF8.9070404@redhat.com> <564E1441.4000604@redhat.com> Message-ID: On 19 November 2015 at 19:26, Bill Burke wrote: > > > On 11/19/2015 9:58 AM, Stian Thorgersen wrote: > >> >> >> On 19 November 2015 at 15:43, Bill Burke > > wrote: >> >> FYI, there's other problems too. Built in clients are looked up via >> their hardcoded ClientIds "admin-console" "broker" etc, again >> because DB ids are generated. >> >> I just don't see how you can remove ID or ClientID. ID is URL >> friendly, ClientID isn't and needs to be encoded. ID is pretty much >> hidden from the user until they want to use the REST interface. If >> you remove ID from the REST interface then ClientID @PathParams are >> gonna have to be encoded which could be a real pain for users, >> especially ones using simple HTTP. While most libraries will encode >> query and form params, users will have to encode by hand other >> mechanisms. >> >> >> For clients I was thinking we should remove ClientID, not ID. We'd allow >> specifying the ID when creating a new client, but not changing it. The >> ID would also have to be URL friendly. >> >> > Removing clientID doesn't work for built-in clients like account, broker, > admin-console, etc. These all need to be located using a predetermined > name. You'd have to figure out an additional alternative to refactor that. Is it not actually bad that they are located using predetermined names? If lookup is on id, you know for a fact that it's the correct client and not just something with the same name. > > > >> There's a similar problem with Groups. Groups are looked up via a >> "path" name "/parent/child" in the REST interface, then you have >> use the ID to reference the rest of the GROUP rest interface i.e. >> groups/{ID}/children. >> >> >> We'll have similar problems with role namespaces if you want to >> incorporate them into the REST interface. >> >> ClientId should be forced to become a URL friendly string (no >> special characters or '/' or "?", etc.). SAML can introduce an >> Issuer URL or something. After that, you can remove ID. That's the >> only solution I can see that will work well. >> >> >> For groups and roles couldn't the ID be the full path? Then the REST >> endpoints would be aware of "hierarchies" so you could just do: >> >> get /auth/realms/groups/group-parent/group-child >> >> And: >> >> get /auth/realms/roles/role/path/to/role/myrole >> >> > I didn't even try to see if that worked. The issue is that there are > additional endpoints after the path: > > .../children > .../composites > > And methods to add remove group children and role composites. I'm pretty > sure there are some JAX-RS and/or regular-expression mapping issues with > that. At least that's what I remember, might be a false memory. Didn't think of the additional stuff under each element. Even if it works it would be messy. > > > > ?? >> >> I think the current approach of having REST endpoints that retrieve >> based on ID is a pain point for a lot of users. They create a client >> with a named client-id or role with a name, then have to use a different >> ID to retrieve from the rest endpoints >> >> > The way it works for groups now is that i have a /group-by-path lookup > mechanism which returns a GroupRepresentation, the id is extracted from > that, then you invoke from there, i.e.: > > GroupRepresentation topGroup = realm.getGroupByPath("/the/top"); > List roles = new LinkedList<>(); > roles.add(topRole); > realm.groups().group(topGroup.getId()).roles().realmLevel().add(roles); > > Alternatively we could add links to the GroupRepresentation like 'self' > that points to the resource endpoint. > I never liked it when we had multiple entry points to the same resource. What about using something like: GET ../users?username=&single=true GET ../users?email=&single=true That returns a single UserRepresentation including 'self' (../users/). Same for groups: GET ../groups?path=&single=true > > > >> >> >> On 11/19/2015 8:59 AM, Stian Thorgersen wrote: >> >> How about: >> >> * IDs are usually generated, but can be specified. It should not >> be >> possible to change them though >> >> * Instead of alias, name and all other ways of setting a human >> readable >> name we add Display Name and Display Class >> >> Not sure how to deal with migration, but if we agree on those >> points, >> then we can figure that out >> >> On 19 November 2015 at 14:49, Bill Burke > >> >> wrote: >> >> One more thing. Another problem is that if you switch to >> ID for >> everything, then demos etc. will no longer run out of the >> box and you'll >> have to modify adapter configuration to set the newly >> generated DB id. >> Really, any piece of configuration that references things >> by client ID >> will no longer work. >> >> So, there's really 2 issues: >> >> * Using ClientID in admin-console URLs doesn't work well as >> the client >> IDs could have '/' characters in them >> * Using ID only messes up pre-defined, pre-packaged adapter >> config and >> you have to edit these files after the keycloak DB is set up. >> >> On 11/19/2015 8:40 AM, Bill Burke wrote: >> > We currently actually have 3 client identifers: ID, >> ClientID, >> and Name. >> > >> > Also, I think there are a lot of duplicate clientID >> names between >> > realms. i.e. "admin-console" "broker", etc... >> > >> > A search by clientID is (realm + clientID). >> > >> > Marek is right about why I switched everything to ID in >> the admin >> > console. For SAML I just didn't want to add yet another >> client >> > identifier since we already had 3. >> > >> > On 11/19/2015 7:52 AM, Marek Posolda wrote: >> >> +1 for this change. >> >> >> >> I am just not sure if we should set the "id" to the >> current value of >> >> "client-id" ? Few things to note: >> >> >> >> - SAML clients currently use clientId in form of URL. >> For example in >> >> SAML demo there are clientIds like >> "http://localhosT:8080/employee-sig" >> >> . I don't know if it's requirement, maybe it's possible >> to solve it >> >> somehow (ie. introduce different attribute for SAML >> client to store >> >> these URLs). But from what I remember, Bill changed admin >> console to use >> >> "id" instead of "clientId" because there were issues >> with URL-like >> >> clientId in admin console . So if we overwrite the "id" >> with current >> >> "client-id" the issue will be back. >> >> >> >> - Migration might be a pain. Many tables (roles, >> protocolMappers, user >> >> consents, offline clientSessions ...) references client >> by "id" . >> >> Overwriting "id" with "client-id" means that we will >> need to >> change all >> >> those DB records. And there are things like foreign >> keys etc... >> >> >> >> Shouldn't do vice-versa and just remove current >> "client-id" and ask >> >> people to update their keycloak.json adapter >> configurations? On the >> >> other hand, removing "client-id" might break migration >> of JSON >> exported >> >> realms as the JSON entities are using "client-id" for >> referencing client. >> >> >> >> It seems the migration will be a pain regardless of >> whatever >> direction >> >> we choose :-( >> >> >> >> Marek >> >> >> >> On 16/11/15 14:54, Stian Thorgersen wrote: >> >>> We have both "id" and "client-id" for clients in >> Keycloak at the >> >>> moment. This seems unnecessary and complex. >> >>> >> >>> The model can retrieve clients on either value. In token >> endpoints the >> >>> "client-id" is used. In admin endpoints the "id" is >> used. >> >>> >> >>> Also, in most cases it would be simpler for users to >> just have a >> >>> generated id than having to come up with one >> themselves. The id >> >>> doesn't have to be human readable either as we have >> name for that. >> >>> >> >>> OpenID Connect expects "client-id" to be generated by >> the IdP and >> >>> can't be changed once created. >> >>> >> >>> I propose we remove "client-id" and only keep id. >> >>> >> >>> For migration of existing clients we would set the >> "id" value >> to the >> >>> current value of "client-id". This would require no >> changes to >> adapter >> >>> configs. When creating new clients from the admin >> console we >> would not >> >>> allow setting the "client-id", instead just display it >> after the >> >>> client was created. When importing clients it would be >> possible >> to set >> >>> the id (and for backwards compatibility we would set >> "id" equal >> to the >> >>> "client-id" field. >> >>> >> >>> >> >>> _______________________________________________ >> >>> keycloak-dev mailing list >> >>> keycloak-dev at lists.jboss.org >> >> > > >> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> >> >> >> >> _______________________________________________ >> >> keycloak-dev mailing list >> >> keycloak-dev at lists.jboss.org >> >> > > >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> >> > > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> >> >> > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/4a53ae99/attachment-0001.html From sthorger at redhat.com Mon Nov 23 04:03:25 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 10:03:25 +0100 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: <5652C63D.7070106@redhat.com> References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> <564F47E4.3010809@redhat.com> <564F4D4A.3050805@redhat.com> <564F5113.9030202@redhat.com> <5652C63D.7070106@redhat.com> Message-ID: We need to support use-cases where OTP is required after login with external IdP. This should be built-in IMO. We should have a easy way to set if OTP is required for a realm or not. Then an option on each IdP to configure if it can skip the OTP requirement or not. Ideally we'd also have an option to check if a user used OTP on the external IdP or not, and require using OTP in Keycloak if not. In the past we had the option of requiring OTP for a realm, but that's now gone and instead we set required on the otp authenticator in the browser flow. However, that's no longer used in brokered logins. This is a regression and needs fixing. On 23 November 2015 at 08:54, Marek Posolda wrote: > On 20/11/15 18:46, Dane Barentine wrote: > > I can look at the IdentityProvider Mapper and see. As you said it's a workaround though and now requires me to create a mapper and an authenticator to handle both types of authentication. > > Regarding this: "Isn't it the more proper option for your usecase to use OTP on the second server side instead?" > > In a lot of cases it would probably be more proper. But it's not always practical as we don't always control the IDPs. So the use case for OTP is if we are using an IDP that either doesn't support, or the team that is running it doesn't want to support, something like OTP but we still want to require it when logging in through Keycloak. The other use case is when we are making authentication/authorization decisions based on some other sort risk assessment or user required action. This may involve using services or code that we will never be able to plug into an IDP. In those cases a post broker login flow would allow us to add on a consistent layer across all the brokered IDPs regardless of what they can technically support. > > You can use requiredAction SPI after broker authentication. That is > triggered after each authentication (classic or broker) . See some docs > here : > http://keycloak.github.io/docs/userguide/keycloak-server/html/auth_spi.html#d4e3420 > . > > Maybe using RequiredAction SPI for OTP authentication is even better than > IdentityProviderMapper (even if it's still seems to be a workaround). In > your RequiredActionProvider.evaluateTriggers(), you will check that user > was authenticated through broker and if yes, you put some requiredAction > like "authenticate_otp" to clientSession. Then in requiredActionChallenge() > you will redirect to OTP form and in processAction() you will check if > OTP was successful. The code in requiredActionChallenge and processAction > methods might be very similar to the code in OTPFormAuthenticator. You > can likely inspire from here. > > If you still have issues, feel free to create JIRA and we will try to look > at improve things in Keycloak. > > Regards, > Marek > > > Dane > > -----Original Message----- > From: keycloak-dev-bounces at lists.jboss.org [mailto:keycloak-dev-bounces at lists.jboss.org ] On Behalf Of Marek Posolda > Sent: Friday, November 20, 2015 8:58 AM > To: Bill Burke ; keycloak-dev at lists.jboss.org > Subject: Re: [keycloak-dev] Identity Broker login flow > > Yes, however the new "First Broker Login" flow I've added is triggered just when there is no Keycloak user already linked to IdentityProvider. > In case that there is already some Keycloak user linked to identityProvider identity, the flow is not executed. Should it be changed to trigger flow after each broker login? I didn't want to do that as there were no use-cases (until now) . And always trigger the flow might require another browser redirect.. > > Marek > > On 20/11/15 17:41, Bill Burke wrote: > > Oh, I thought our broker flows happened after the user logged into the > external IDP? > > On 11/20/2015 11:18 AM, Marek Posolda wrote: > > Yep. I know :-\ > > As usually, the most proper solution requires most of the changes and > refactoring... > > For this particular case, I wonder if Dane can use the approach with > implement IdentityProviderMapper to redirect to OTP login. This is > workaround though (IdentityProviderMapper is likely not designed for > the usecases like this), however I think it should work. This won't > require any changes in existing Keycloak code. > > I can introduce "post-broker-login" flow, but not sure if it's > something we should support OOTB. I don't know if it's useful for > more people. So I would rather skip adding post-broker-login as "authentication levels" > will be more proper and more generic solution IMO. > > Marek > > > On 20/11/15 16:05, Bill Burke wrote: > > All something we want to do, but do we have time? Very soon we need > to start focusing on cleaning up the APIs/SPIs, documentation, > improving UI, performance, and the tons of minor problems people are > experiencing. > > On 11/20/2015 3:45 AM, Marek Posolda wrote: > > I wonder if we should address this together with reducing number of > redirects ( https://issues.jboss.org/browse/KEYCLOAK-2098 ) and > also support for "authentication levels" . We can encapsulate all > the state of authentication in ClientSession. If someone refreshes > the page, we will retrieve the ClientSession from the code and the > current state of execution will be retrieved from ClientSession itself. > > ClientSession will also encapsulate all the state of authentication > after authentication is finished. For example there will be > "cookie" if ClientSession was authenticated through cookie, > "password, otp" if clientSession was authenticated through password > + otp or "broker-facebook" if it was authenticated through > Facebook. There will be also timestamps of when each successful > authentication happened. > The > state will be partially saved in UserSession, so if clientSession > was authenticated through cookie, it can retrieve from userSession > when the "real" authentication happened etc. > > This will allow us to support periodic re-authentication, > authentication levels and different requirements for > "authentication level" for individual application. For example at > each SSO login or refresh, we will be able to: > - ask user for re-authenticate through OTP every 30 minutes > - ask user for re-authenticate through OTP if he was authenticated > through Facebook broker > - ask user for re-authenticate through OTP if application required > that with some magic parameter (AFAIK both OIDC or SAML has some > parameters for specify authentication levels) > > Marek > > On 20/11/15 08:41, Marek Posolda wrote: > > You're right, we don't have this right now. I don't know if it's > something we should support OOTB. The idea of broker login is, > that you delegate authentication to another SSO/social server. > Once the second server say "Ok, user is authenticated", we treat > him as authenticated on Keycloak side too. Isn't it the more > proper option for your usecase to use OTP on the second server side instead? > > Another option is to implement IdentityProviderMapper and in > "updateBrokeredUser" method, you will redirect user to OTP login. > Could you try this? > > > If we want to support another login flow triggered after each > broker login (which I am not convinced TBH), we can either: > > 1) Introduce "post-broker-login" flow, which will be configurable > per IdentityProvider. By default, it will be empty . > > 2) Use just one flow, which will be triggered after each broker > login (current "first broker login" flow is triggered only if > federatedIdentity doesn't yet exist in Keycloak). In this case, > the current "first broker login" flow will need to be renamed to > "broker login" and more logic will need to be moved from > IdentityBrokerService to the flow itself. The disadvantage of this > option is, that it may always require another redirect to trigger > authentication flows. But we're trying to reduce the number of > redirects (https://issues.jboss.org/browse/KEYCLOAK-2098 ) > > Marek > > On 20/11/15 00:06, Dane Barentine wrote: > > Hi all, > > I'm trying to add a custom authenticator and it appears that that > there is no way to insert it in the flow if it's a brokered IDP > login where the linked Keycloak account already exists. > > If it's a local Keycloak user I can use the Browser flow and if > it's a new brokered user the First Broker Login flow will > execute. But I don't see a flow that would allow me to insert > something like OTP after a brokered login of an existing user. > > If I'm just missing it let me know but I think there needs to be > some sort of flow for brokered logins that runs on both existing > and new users. For new users it would run after the First Broker > Login flow. > Or better yet maybe a flow that would allow things such as OTP to > happen after any brokered or local login. That way it wouldn't > have to be configured in multiple flows. > > Thanks > > Dane > > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/7ca77f16/attachment.html From sthorger at redhat.com Mon Nov 23 04:04:46 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 10:04:46 +0100 Subject: [keycloak-dev] no empty password in UserFederationProvider In-Reply-To: <5652C806.4090502@redhat.com> References: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> <564F4D06.2070205@redhat.com> <564F4F46.5090700@redhat.com> <5652C806.4090502@redhat.com> Message-ID: Question: a user passing an empty password is that really a brute force attempt? Is it not more likely that a user has forgotten to enter the password? I'm asking because I'm not convinced forgetting to enter a value in the password field should count as a login attempt. On 23 November 2015 at 09:02, Marek Posolda wrote: > Sure, that would be fine. But again, it's important that with default > settings, LDAPOperationManager.authenticate won't automatically > authenticate AD users with empty password (which may happen when > anonymous bind is enabled on AD side). > > Marek > > On 20/11/15 18:22, Michael Gerber wrote: > > We?ve got a custom UserFederationProvider, which authenticate users > against an AD or DB. Therefore, we need to know if a user entered an empty > password. > > I will create a PR and jira ticket for that, ok? > > > >> On 20.11.2015, at 17:50, Marek Posolda wrote: > >> > >> That will be the easiest path to use our BruteForceProtector. > >> > >> However AD also has some "BruteForceProtector" of it's own, which > disables user in AD when he reach some count of invalid attempts. And I > guess Michael wants to use that one and disable user in AD as well. > >> > >> Marek > >> > >> On 20/11/15 17:40, Bill Burke wrote: > >>> You can I guess, but why does it matter? invalidPassword hits the > brute > >>> force detector if it is turned on. > >>> > >>> On 11/20/2015 10:16 AM, Michael Gerber wrote: > >>>> AbstractUsernameFormAuthenticator.validatePassword > >>>> > >>>> public boolean validatePassword(AuthenticationFlowContext context, > UserModel user, MultivaluedMap inputData) { > >>>> List credentials =new LinkedList<>(); > >>>> String password = > inputData.getFirst(CredentialRepresentation.PASSWORD); > >>>> if (password ==null || password.isEmpty()) { > >>>> invalidPassword(context, user); > >>>> return false; > >>>> } > >>>> credentials.add(UserCredentialModel.password(password)); > >>>> boolean valid = > context.getSession().users().validCredentials(context.getRealm(), user, > credentials); > >>>> if (!valid) { > >>>> invalidPassword(context, user); > >>>> return false; > >>>> } > >>>> return true; > >>>> } > >>>> > >>>> I think we can remove the first if (password == null || > password.isEmpty()) > >>>> > >>>> Am 20. November 2015 um 16:11 schrieb Bill Burke : > >>>> > >>>>> Point me to the code? > >>>>> > >>>>> On 11/20/2015 9:04 AM, Michael Gerber wrote: > >>>>>> Hi All, > >>>>>> > >>>>>> keycloak does not pass an empty password to the validCredentials > method > >>>>>> in the UserFederationProvider class. > >>>>>> Is there a reason for that? I would like to authenticate against an > AD > >>>>>> even if the password is empty, otherwise the user won't be blocked > after > >>>>>> x attempts. > >>>>>> > >>>>>> Michael > >>>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>>> keycloak-dev mailing list > >>>>>> keycloak-dev at lists.jboss.org > >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>>>>> > >>>>> -- > >>>>> Bill Burke > >>>>> JBoss, a division of Red Hat > >>>>> http://bill.burkecentral.com > >>>>> _______________________________________________ > >>>>> keycloak-dev mailing list > >>>>> keycloak-dev at lists.jboss.org > >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/1b8c6a29/attachment-0001.html From sthorger at redhat.com Mon Nov 23 05:43:27 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 11:43:27 +0100 Subject: [keycloak-dev] Fixing bugs? Add tests! Message-ID: If you are fixing a bug please also add a test. This makes sure the issue is actually resolved and also that we don't have a regression in the future. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/f03127f5/attachment.html From gerbermichi at me.com Mon Nov 23 08:33:26 2015 From: gerbermichi at me.com (Michael Gerber) Date: Mon, 23 Nov 2015 13:33:26 +0000 Subject: [keycloak-dev] no empty password in UserFederationProvider In-Reply-To: References: <57a63bb6-94b0-44b0-b485-4d563fdbbdd7@me.com> <564F4D06.2070205@redhat.com> <564F4F46.5090700@redhat.com> <5652C806.4090502@redhat.com> Message-ID: <2E1C51A1-763B-4926-B745-E8CFE20FDF29@me.com> I think it?s the responsibility of the UserFederationProvider to decide if a password is valid or not. Therefore, even empty passwords have to be send to it. > On 23.11.2015, at 09:04, Stian Thorgersen wrote: > > Question: a user passing an empty password is that really a brute force attempt? Is it not more likely that a user has forgotten to enter the password? I'm asking because I'm not convinced forgetting to enter a value in the password field should count as a login attempt. > > On 23 November 2015 at 09:02, Marek Posolda > wrote: > Sure, that would be fine. But again, it's important that with default > settings, LDAPOperationManager.authenticate won't automatically > authenticate AD users with empty password (which may happen when > anonymous bind is enabled on AD side). > > Marek > > On 20/11/15 18:22, Michael Gerber wrote: > > We?ve got a custom UserFederationProvider, which authenticate users against an AD or DB. Therefore, we need to know if a user entered an empty password. > > I will create a PR and jira ticket for that, ok? > > > >> On 20.11.2015, at 17:50, Marek Posolda > wrote: > >> > >> That will be the easiest path to use our BruteForceProtector. > >> > >> However AD also has some "BruteForceProtector" of it's own, which disables user in AD when he reach some count of invalid attempts. And I guess Michael wants to use that one and disable user in AD as well. > >> > >> Marek > >> > >> On 20/11/15 17:40, Bill Burke wrote: > >>> You can I guess, but why does it matter? invalidPassword hits the brute > >>> force detector if it is turned on. > >>> > >>> On 11/20/2015 10:16 AM, Michael Gerber wrote: > >>>> AbstractUsernameFormAuthenticator.validatePassword > >>>> > >>>> public boolean validatePassword(AuthenticationFlowContext context, UserModel user, MultivaluedMap inputData) { > >>>> List credentials =new LinkedList<>(); > >>>> String password = inputData.getFirst(CredentialRepresentation.PASSWORD); > >>>> if (password ==null || password.isEmpty()) { > >>>> invalidPassword(context, user); > >>>> return false; > >>>> } > >>>> credentials.add(UserCredentialModel.password(password)); > >>>> boolean valid = context.getSession().users().validCredentials(context.getRealm(), user, credentials); > >>>> if (!valid) { > >>>> invalidPassword(context, user); > >>>> return false; > >>>> } > >>>> return true; > >>>> } > >>>> > >>>> I think we can remove the first if (password == null || password.isEmpty()) > >>>> > >>>> Am 20. November 2015 um 16:11 schrieb Bill Burke >: > >>>> > >>>>> Point me to the code? > >>>>> > >>>>> On 11/20/2015 9:04 AM, Michael Gerber wrote: > >>>>>> Hi All, > >>>>>> > >>>>>> keycloak does not pass an empty password to the validCredentials method > >>>>>> in the UserFederationProvider class. > >>>>>> Is there a reason for that? I would like to authenticate against an AD > >>>>>> even if the password is empty, otherwise the user won't be blocked after > >>>>>> x attempts. > >>>>>> > >>>>>> Michael > >>>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>>> keycloak-dev mailing list > >>>>>> keycloak-dev at lists.jboss.org > > >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>>>>> > >>>>> -- > >>>>> Bill Burke > >>>>> JBoss, a division of Red Hat > >>>>> http://bill.burkecentral.com > >>>>> _______________________________________________ > >>>>> keycloak-dev mailing list > >>>>> keycloak-dev at lists.jboss.org > > >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/1811f644/attachment.html From bburke at redhat.com Mon Nov 23 09:06:27 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 23 Nov 2015 09:06:27 -0500 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> <564DDFF8.9070404@redhat.com> <564E1441.4000604@redhat.com> Message-ID: <56531D63.9080904@redhat.com> On 11/23/2015 3:19 AM, Stian Thorgersen wrote: > Removing clientID doesn't work for built-in clients like account, > broker, admin-console, etc. These all need to be located using a > predetermined name. You'd have to figure out an additional > alternative to refactor that. > > > Is it not actually bad that they are located using predetermined names? > If lookup is on id, you know for a fact that it's the correct client and > not just something with the same name. > Remember, the predetermined names are "account", "broker" etc. These are non-unique names. They have to be predetermined or at least indexed in a realm attribute by a predetermined name. > > I never liked it when we had multiple entry points to the same resource. > What about using something like: > > GET ../users?username=&single=true > GET ../users?email=&single=true > > That returns a single UserRepresentation including 'self' > (../users/). > > Same for groups: > > GET ../groups?path=&single=true > That works too. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Mon Nov 23 09:07:34 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 23 Nov 2015 09:07:34 -0500 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: <5652C63D.7070106@redhat.com> References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> <564F47E4.3010809@redhat.com> <564F4D4A.3050805@redhat.com> <564F5113.9030202@redhat.com> <5652C63D.7070106@redhat.com> Message-ID: <56531DA6.1060508@redhat.com> On 11/23/2015 2:54 AM, Marek Posolda wrote: > On 20/11/15 18:46, Dane Barentine wrote: >> I can look at the IdentityProvider Mapper and see. As you said it's a workaround though and now requires me to create a mapper and an authenticator to handle both types of authentication. >> >> Regarding this: "Isn't it the more proper option for your usecase to use OTP on the second server side instead?" >> >> In a lot of cases it would probably be more proper. But it's not always practical as we don't always control the IDPs. So the use case for OTP is if we are using an IDP that either doesn't support, or the team that is running it doesn't want to support, something like OTP but we still want to require it when logging in through Keycloak. The other use case is when we are making authentication/authorization decisions based on some other sort risk assessment or user required action. This may involve using services or code that we will never be able to plug into an IDP. In those cases a post broker login flow would allow us to add on a consistent layer across all the brokered IDPs regardless of what they can technically support. > You can use requiredAction SPI after broker authentication. That is > triggered after each authentication (classic or broker) . See some docs > here : > http://keycloak.github.io/docs/userguide/keycloak-server/html/auth_spi.html#d4e3420 > . > > Maybe using RequiredAction SPI for OTP authentication is even better > than IdentityProviderMapper (even if it's still seems to be a > workaround). In your RequiredActionProvider.evaluateTriggers(), you will > check that user was authenticated through broker and if yes, you put > some requiredAction like "authenticate_otp" to clientSession. Then in > requiredActionChallenge() you will redirect to OTP form and in > processAction() you will check if OTP was successful. The code in > requiredActionChallenge and processAction methods might be very similar > to the code in OTPFormAuthenticator. You can likely inspire from here. > > If you still have issues, feel free to create JIRA and we will try to > look at improve things in Keycloak. > It would be better to have a post-login flow. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Mon Nov 23 09:08:24 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 23 Nov 2015 09:08:24 -0500 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> <564F47E4.3010809@redhat.com> <564F4D4A.3050805@redhat.com> <564F5113.9030202@redhat.com> <5652C63D.7070106@redhat.com> Message-ID: <56531DD8.7070009@redhat.com> We need a full post-IDP flow. Marek just added a flow that if it was a new user. On 11/23/2015 4:03 AM, Stian Thorgersen wrote: > We need to support use-cases where OTP is required after login with > external IdP. This should be built-in IMO. We should have a easy way to > set if OTP is required for a realm or not. Then an option on each IdP to > configure if it can skip the OTP requirement or not. Ideally we'd also > have an option to check if a user used OTP on the external IdP or not, > and require using OTP in Keycloak if not. > > In the past we had the option of requiring OTP for a realm, but that's > now gone and instead we set required on the otp authenticator in the > browser flow. However, that's no longer used in brokered logins. This is > a regression and needs fixing. > > On 23 November 2015 at 08:54, Marek Posolda > wrote: > > On 20/11/15 18:46, Dane Barentine wrote: >> I can look at the IdentityProvider Mapper and see. As you said it's a workaround though and now requires me to create a mapper and an authenticator to handle both types of authentication. >> >> Regarding this: "Isn't it the more proper option for your usecase to use OTP on the second server side instead?" >> >> In a lot of cases it would probably be more proper. But it's not always practical as we don't always control the IDPs. So the use case for OTP is if we are using an IDP that either doesn't support, or the team that is running it doesn't want to support, something like OTP but we still want to require it when logging in through Keycloak. The other use case is when we are making authentication/authorization decisions based on some other sort risk assessment or user required action. This may involve using services or code that we will never be able to plug into an IDP. In those cases a post broker login flow would allow us to add on a consistent layer across all the brokered IDPs regardless of what they can technically support. > You can use requiredAction SPI after broker authentication. That is > triggered after each authentication (classic or broker) . See some > docs here : > http://keycloak.github.io/docs/userguide/keycloak-server/html/auth_spi.html#d4e3420 > . > > Maybe using RequiredAction SPI for OTP authentication is even better > than IdentityProviderMapper (even if it's still seems to be a > workaround). In your RequiredActionProvider.evaluateTriggers(), you > will check that user was authenticated through broker and if yes, > you put some requiredAction like "authenticate_otp" to > clientSession. Then in requiredActionChallenge() you will redirect > to OTP form and in processAction() you will check if OTP was > successful. The code in requiredActionChallenge and processAction > methods might be very similar to the code in OTPFormAuthenticator. > You can likely inspire from here. > > If you still have issues, feel free to create JIRA and we will try > to look at improve things in Keycloak. > > Regards, > Marek >> Dane >> >> -----Original Message----- >> From:keycloak-dev-bounces at lists.jboss.org >> [mailto:keycloak-dev-bounces at lists.jboss.org] On Behalf Of Marek Posolda >> Sent: Friday, November 20, 2015 8:58 AM >> To: Bill Burke ;keycloak-dev at lists.jboss.org >> Subject: Re: [keycloak-dev] Identity Broker login flow >> >> Yes, however the new "First Broker Login" flow I've added is triggered just when there is no Keycloak user already linked to IdentityProvider. >> In case that there is already some Keycloak user linked to identityProvider identity, the flow is not executed. Should it be changed to trigger flow after each broker login? I didn't want to do that as there were no use-cases (until now) . And always trigger the flow might require another browser redirect.. >> >> Marek >> >> On 20/11/15 17:41, Bill Burke wrote: >>> Oh, I thought our broker flows happened after the user logged into the >>> external IDP? >>> >>> On 11/20/2015 11:18 AM, Marek Posolda wrote: >>>> Yep. I know :-\ >>>> >>>> As usually, the most proper solution requires most of the changes and >>>> refactoring... >>>> >>>> For this particular case, I wonder if Dane can use the approach with >>>> implement IdentityProviderMapper to redirect to OTP login. This is >>>> workaround though (IdentityProviderMapper is likely not designed for >>>> the usecases like this), however I think it should work. This won't >>>> require any changes in existing Keycloak code. >>>> >>>> I can introduce "post-broker-login" flow, but not sure if it's >>>> something we should support OOTB. I don't know if it's useful for >>>> more people. So I would rather skip adding post-broker-login as "authentication levels" >>>> will be more proper and more generic solution IMO. >>>> >>>> Marek >>>> >>>> >>>> On 20/11/15 16:05, Bill Burke wrote: >>>>> All something we want to do, but do we have time? Very soon we need >>>>> to start focusing on cleaning up the APIs/SPIs, documentation, >>>>> improving UI, performance, and the tons of minor problems people are >>>>> experiencing. >>>>> >>>>> On 11/20/2015 3:45 AM, Marek Posolda wrote: >>>>>> I wonder if we should address this together with reducing number of >>>>>> redirects (https://issues.jboss.org/browse/KEYCLOAK-2098 ) and >>>>>> also support for "authentication levels" . We can encapsulate all >>>>>> the state of authentication in ClientSession. If someone refreshes >>>>>> the page, we will retrieve the ClientSession from the code and the >>>>>> current state of execution will be retrieved from ClientSession itself. >>>>>> >>>>>> ClientSession will also encapsulate all the state of authentication >>>>>> after authentication is finished. For example there will be >>>>>> "cookie" if ClientSession was authenticated through cookie, >>>>>> "password, otp" if clientSession was authenticated through password >>>>>> + otp or "broker-facebook" if it was authenticated through >>>>>> Facebook. There will be also timestamps of when each successful >>>>>> authentication happened. >>>>>> The >>>>>> state will be partially saved in UserSession, so if clientSession >>>>>> was authenticated through cookie, it can retrieve from userSession >>>>>> when the "real" authentication happened etc. >>>>>> >>>>>> This will allow us to support periodic re-authentication, >>>>>> authentication levels and different requirements for >>>>>> "authentication level" for individual application. For example at >>>>>> each SSO login or refresh, we will be able to: >>>>>> - ask user for re-authenticate through OTP every 30 minutes >>>>>> - ask user for re-authenticate through OTP if he was authenticated >>>>>> through Facebook broker >>>>>> - ask user for re-authenticate through OTP if application required >>>>>> that with some magic parameter (AFAIK both OIDC or SAML has some >>>>>> parameters for specify authentication levels) >>>>>> >>>>>> Marek >>>>>> >>>>>> On 20/11/15 08:41, Marek Posolda wrote: >>>>>>> You're right, we don't have this right now. I don't know if it's >>>>>>> something we should support OOTB. The idea of broker login is, >>>>>>> that you delegate authentication to another SSO/social server. >>>>>>> Once the second server say "Ok, user is authenticated", we treat >>>>>>> him as authenticated on Keycloak side too. Isn't it the more >>>>>>> proper option for your usecase to use OTP on the second server side instead? >>>>>>> >>>>>>> Another option is to implement IdentityProviderMapper and in >>>>>>> "updateBrokeredUser" method, you will redirect user to OTP login. >>>>>>> Could you try this? >>>>>>> >>>>>>> >>>>>>> If we want to support another login flow triggered after each >>>>>>> broker login (which I am not convinced TBH), we can either: >>>>>>> >>>>>>> 1) Introduce "post-broker-login" flow, which will be configurable >>>>>>> per IdentityProvider. By default, it will be empty . >>>>>>> >>>>>>> 2) Use just one flow, which will be triggered after each broker >>>>>>> login (current "first broker login" flow is triggered only if >>>>>>> federatedIdentity doesn't yet exist in Keycloak). In this case, >>>>>>> the current "first broker login" flow will need to be renamed to >>>>>>> "broker login" and more logic will need to be moved from >>>>>>> IdentityBrokerService to the flow itself. The disadvantage of this >>>>>>> option is, that it may always require another redirect to trigger >>>>>>> authentication flows. But we're trying to reduce the number of >>>>>>> redirects ( >>>>>>> https://issues.jboss.org/browse/KEYCLOAK-2098 ) >>>>>>> >>>>>>> Marek >>>>>>> >>>>>>> On 20/11/15 00:06, Dane Barentine wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> I'm trying to add a custom authenticator and it appears that that >>>>>>>> there is no way to insert it in the flow if it's a brokered IDP >>>>>>>> login where the linked Keycloak account already exists. >>>>>>>> >>>>>>>> If it's a local Keycloak user I can use the Browser flow and if >>>>>>>> it's a new brokered user the First Broker Login flow will >>>>>>>> execute. But I don't see a flow that would allow me to insert >>>>>>>> something like OTP after a brokered login of an existing user. >>>>>>>> >>>>>>>> If I'm just missing it let me know but I think there needs to be >>>>>>>> some sort of flow for brokered logins that runs on both existing >>>>>>>> and new users. For new users it would run after the First Broker >>>>>>>> Login flow. >>>>>>>> Or better yet maybe a flow that would allow things such as OTP to >>>>>>>> happen after any brokered or local login. That way it wouldn't >>>>>>>> have to be configured in multiple flows. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Dane >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> keycloak-dev mailing list >>>>>>>> keycloak-dev at lists.jboss.org >>>>>>>> >>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> _______________________________________________ >>>>>>> keycloak-dev mailing list >>>>>>> keycloak-dev at lists.jboss.org >>>>>>> >>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Mon Nov 23 09:18:09 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 15:18:09 +0100 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: <56531D63.9080904@redhat.com> References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> <564DDFF8.9070404@redhat.com> <564E1441.4000604@redhat.com> <56531D63.9080904@redhat.com> Message-ID: On 23 November 2015 at 15:06, Bill Burke wrote: > > > On 11/23/2015 3:19 AM, Stian Thorgersen wrote: > >> Removing clientID doesn't work for built-in clients like account, >> broker, admin-console, etc. These all need to be located using a >> predetermined name. You'd have to figure out an additional >> alternative to refactor that. >> >> >> Is it not actually bad that they are located using predetermined names? >> If lookup is on id, you know for a fact that it's the correct client and >> not just something with the same name. >> >> > Remember, the predetermined names are "account", "broker" etc. These are > non-unique names. They have to be predetermined or at least indexed in a > realm attribute by a predetermined name. My thinking was we'd add realm attributes > > > >> I never liked it when we had multiple entry points to the same resource. >> What about using something like: >> >> GET ../users?username=&single=true >> GET ../users?email=&single=true >> >> That returns a single UserRepresentation including 'self' >> (../users/). >> >> Same for groups: >> >> GET ../groups?path=&single=true >> >> > That works too. It's slightly more elegant isn't it? As everything has 1 place in the endpoints, rather than multiple. With a 'search' option under 'users' we'd only have: /users But, with the alternative approach (users-by-username, group-by-path, etc) we end up with multiple locations: /users /users-by-email /users-by-username Which I think is harder to use + not as nice for audit/logging > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/528b371f/attachment.html From ayoung at redhat.com Mon Nov 23 09:22:37 2015 From: ayoung at redhat.com (Adam Young) Date: Mon, 23 Nov 2015 09:22:37 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <56528D11.3010308@redhat.com> References: <5652763D.3050304@redhat.com> <56528D11.3010308@redhat.com> Message-ID: <5653212D.8010706@redhat.com> On 11/22/2015 10:50 PM, Bill Burke wrote: > Most Java EE apps, simple permission stuff is handled by the > applications themselves. The security layer propagates identity and > user/role mappings and the application decides which roles have > permission to access which resource. So, Keycloak core can focus on > provide user/group/role/attribute authentication and information. The > service Pedro is working on will offer an optional, complimentary way to > centrally manage permissions if you want to. My on personal opinion is > that most apps will want to manage resource permissions in their own > special app-specific way and Pedro's stuff will be most useful as a > backend-service. So the analogue in Keystone is the Policy stuff. Again, Keystone has had to weather a storm; Access Control permissions in OpenStack are done at the Python API (not URL) level. The code calls out to the policy API to check that the context of the call matches the attributes of the resource. Practically speaking, this means two checks: does the role match, and does the scope (project or user) match. It would be easier for deployers if it was URL based. Instead, it is deep in the code, and the reason being that often an application needs to fetch an object from the database to check ownership before checking scope (what project owns the resource). What we learned is that this can and should actually be two checks. The role check can be done in a non-application specific way, and performed in the middleware stack; kinda like in the Tomcat Webserver prior to hitting the servlet itself, but in a python WSGI way instead. This part could and should be dynamically fetched from the Keystone server. Only the scope check needs to be done in a resource specific way. There is support at the EJB/Hibernate layer for this kind of access control, too, and that may be a better place to focus on completely. I suspect it is this kind of "Dynamic Policy" stuff that Pedro is working on. Do we have docs for that? I'd love to compare notes. I'm in the US, on East Coast time. THis is a short Week due to Thanksgiving, but I'd love to have a video conference of some sort the following week to sync up the lessons learned from both projects. From bburke at redhat.com Mon Nov 23 09:32:08 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 23 Nov 2015 09:32:08 -0500 Subject: [keycloak-dev] Client ID and Client ClientID - I propose we remove one In-Reply-To: References: <564DC62B.7020205@redhat.com> <564DD15A.9080907@redhat.com> <564DD34F.603@redhat.com> <564DDFF8.9070404@redhat.com> <564E1441.4000604@redhat.com> <56531D63.9080904@redhat.com> Message-ID: <56532368.6000707@redhat.com> On 11/23/2015 9:18 AM, Stian Thorgersen wrote: > It's slightly more elegant isn't it? As everything has 1 place in the > endpoints, rather than multiple. With a 'search' option under 'users' > we'd only have: > > /users > > But, with the alternative approach (users-by-username, group-by-path, > etc) we end up with multiple locations: > > /users > /users-by-email > /users-by-username > > Which I think is harder to use + not as nice for audit/logging > Yeah, that was just a dumb approach (XXX-by-name), apologies, I wasn't thinking. It kind of came about because we kept switching between name and id for lookup. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Mon Nov 23 09:32:51 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 15:32:51 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <5653212D.8010706@redhat.com> References: <5652763D.3050304@redhat.com> <56528D11.3010308@redhat.com> <5653212D.8010706@redhat.com> Message-ID: Pedro is currently away on holiday. I believe he's back in two weeks. We should have an early alpha ready for the authorization services shortly after he's back. You can check out the code at https://github.com/pedroigor/keycloak-authz, but AFAIK it's not quite ready for consumption. The aim of the alpha is to have something available for others to play with and to obtain feedback. It's still very much open to be discussed. Your feedback would be very welcome, but we should wait until Pedro is back before we continue the discussion. On 23 November 2015 at 15:22, Adam Young wrote: > On 11/22/2015 10:50 PM, Bill Burke wrote: > > Most Java EE apps, simple permission stuff is handled by the > > applications themselves. The security layer propagates identity and > > user/role mappings and the application decides which roles have > > permission to access which resource. So, Keycloak core can focus on > > provide user/group/role/attribute authentication and information. The > > service Pedro is working on will offer an optional, complimentary way to > > centrally manage permissions if you want to. My on personal opinion is > > that most apps will want to manage resource permissions in their own > > special app-specific way and Pedro's stuff will be most useful as a > > backend-service. > So the analogue in Keystone is the Policy stuff. Again, Keystone has > had to weather a storm; > > Access Control permissions in OpenStack are done at the Python API (not > URL) level. The code calls out to the policy API to check that the > context of the call matches the attributes of the resource. Practically > speaking, this means two checks: does the role match, and does the > scope (project or user) match. > > > It would be easier for deployers if it was URL based. Instead, it is > deep in the code, and the reason being that often an application needs > to fetch an object from the database to check ownership before checking > scope (what project owns the resource). What we learned is that this > can and should actually be two checks. The role check can be done in a > non-application specific way, and performed in the middleware stack; > kinda like in the Tomcat Webserver prior to hitting the servlet itself, > but in a python WSGI way instead. This part could and should be > dynamically fetched from the Keystone server. Only the scope check > needs to be done in a resource specific way. There is support at the > EJB/Hibernate layer for this kind of access control, too, and that may > be a better place to focus on completely. I suspect it is this kind of > "Dynamic Policy" stuff that Pedro is working on. Do we have docs for > that? I'd love to compare notes. > > I'm in the US, on East Coast time. THis is a short Week due to > Thanksgiving, but I'd love to have a video conference of some sort the > following week to sync up the lessons learned from both projects. > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/1f497be4/attachment.html From sthorger at redhat.com Mon Nov 23 09:36:10 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 15:36:10 +0100 Subject: [keycloak-dev] Issues with Travis Message-ID: There's some issues with Travis and it keeps failing PRs even though the tests pass. I'm looking at it and will update once it's resolved. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/a90230ec/attachment.html From bburke at redhat.com Mon Nov 23 09:40:50 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 23 Nov 2015 09:40:50 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <5653212D.8010706@redhat.com> References: <5652763D.3050304@redhat.com> <56528D11.3010308@redhat.com> <5653212D.8010706@redhat.com> Message-ID: <56532572.5050801@redhat.com> On 11/23/2015 9:22 AM, Adam Young wrote: > On 11/22/2015 10:50 PM, Bill Burke wrote: >> Most Java EE apps, simple permission stuff is handled by the >> applications themselves. The security layer propagates identity and >> user/role mappings and the application decides which roles have >> permission to access which resource. So, Keycloak core can focus on >> provide user/group/role/attribute authentication and information. The >> service Pedro is working on will offer an optional, complimentary way to >> centrally manage permissions if you want to. My on personal opinion is >> that most apps will want to manage resource permissions in their own >> special app-specific way and Pedro's stuff will be most useful as a >> backend-service. > So the analogue in Keystone is the Policy stuff. Again, Keystone has > had to weather a storm; > > Access Control permissions in OpenStack are done at the Python API (not > URL) level. The code calls out to the policy API to check that the > context of the call matches the attributes of the resource. Practically > speaking, this means two checks: does the role match, and does the > scope (project or user) match. > We really should have a BJ session. Basic Java EE does permission via URL. Pedro's stuff will be a lot more abstract. You would submit your parameters (user, group, role, resource attributes, etc.) and get back authorization permissions. All through the UMA protocol. URL is just one type of resource in Pedro's stuff...Pedro should answer though. Stian and I really only have a high level understanding of what he is doing. > > It would be easier for deployers if it was URL based. Instead, it is > deep in the code, and the reason being that often an application needs > to fetch an object from the database to check ownership before checking > scope (what project owns the resource). What we learned is that this > can and should actually be two checks. The role check can be done in a > non-application specific way, and performed in the middleware stack; We don't know shit about the Python stack. Personally I haven't touched Python in 15 years. But anything Java based we have a good pulse on. We've focused mostly on basic Java EE authorization. Most of our users are interested in Keycloak out of the box, or customizing the federation and/or authentication flows. > kinda like in the Tomcat Webserver prior to hitting the servlet itself, > but in a python WSGI way instead. This part could and should be > dynamically fetched from the Keystone server. Only the scope check > needs to be done in a resource specific way. There is support at the > EJB/Hibernate layer for this kind of access control, too, and that may > be a better place to focus on completely. I suspect it is this kind of > "Dynamic Policy" stuff that Pedro is working on. Do we have docs for > that? I'd love to compare notes. > > I'm in the US, on East Coast time. THis is a short Week due to > Thanksgiving, but I'd love to have a video conference of some sort the > following week to sync up the lessons learned from both projects. Yes! Let's figure something out. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Mon Nov 23 09:43:45 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 15:43:45 +0100 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <56532572.5050801@redhat.com> References: <5652763D.3050304@redhat.com> <56528D11.3010308@redhat.com> <5653212D.8010706@redhat.com> <56532572.5050801@redhat.com> Message-ID: +1 Let's all have a hangout when Pedro returns :) On 23 November 2015 at 15:40, Bill Burke wrote: > > > On 11/23/2015 9:22 AM, Adam Young wrote: > > On 11/22/2015 10:50 PM, Bill Burke wrote: > >> Most Java EE apps, simple permission stuff is handled by the > >> applications themselves. The security layer propagates identity and > >> user/role mappings and the application decides which roles have > >> permission to access which resource. So, Keycloak core can focus on > >> provide user/group/role/attribute authentication and information. The > >> service Pedro is working on will offer an optional, complimentary way to > >> centrally manage permissions if you want to. My on personal opinion is > >> that most apps will want to manage resource permissions in their own > >> special app-specific way and Pedro's stuff will be most useful as a > >> backend-service. > > So the analogue in Keystone is the Policy stuff. Again, Keystone has > > had to weather a storm; > > > > Access Control permissions in OpenStack are done at the Python API (not > > URL) level. The code calls out to the policy API to check that the > > context of the call matches the attributes of the resource. Practically > > speaking, this means two checks: does the role match, and does the > > scope (project or user) match. > > > > We really should have a BJ session. Basic Java EE does permission via > URL. Pedro's stuff will be a lot more abstract. You would submit your > parameters (user, group, role, resource attributes, etc.) and get back > authorization permissions. All through the UMA protocol. URL is just > one type of resource in Pedro's stuff...Pedro should answer though. > Stian and I really only have a high level understanding of what he is > doing. > > > > > It would be easier for deployers if it was URL based. Instead, it is > > deep in the code, and the reason being that often an application needs > > to fetch an object from the database to check ownership before checking > > scope (what project owns the resource). What we learned is that this > > can and should actually be two checks. The role check can be done in a > > non-application specific way, and performed in the middleware stack; > > We don't know shit about the Python stack. Personally I haven't touched > Python in 15 years. But anything Java based we have a good pulse on. > We've focused mostly on basic Java EE authorization. Most of our users > are interested in Keycloak out of the box, or customizing the federation > and/or authentication flows. > > > > kinda like in the Tomcat Webserver prior to hitting the servlet itself, > > but in a python WSGI way instead. This part could and should be > > dynamically fetched from the Keystone server. Only the scope check > > needs to be done in a resource specific way. There is support at the > > EJB/Hibernate layer for this kind of access control, too, and that may > > be a better place to focus on completely. I suspect it is this kind of > > "Dynamic Policy" stuff that Pedro is working on. Do we have docs for > > that? I'd love to compare notes. > > > > I'm in the US, on East Coast time. THis is a short Week due to > > Thanksgiving, but I'd love to have a video conference of some sort the > > following week to sync up the lessons learned from both projects. > > Yes! Let's figure something out. > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/6fc4ed45/attachment.html From mposolda at redhat.com Mon Nov 23 09:50:19 2015 From: mposolda at redhat.com (Marek Posolda) Date: Mon, 23 Nov 2015 15:50:19 +0100 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> <564F47E4.3010809@redhat.com> <564F4D4A.3050805@redhat.com> <564F5113.9030202@redhat.com> <5652C63D.7070106@redhat.com> Message-ID: <565327AB.2030000@redhat.com> On 23/11/15 10:03, Stian Thorgersen wrote: > We need to support use-cases where OTP is required after login with > external IdP. This should be built-in IMO. We should have a easy way > to set if OTP is required for a realm or not. Then an option on each > IdP to configure if it can skip the OTP requirement or not. Ideally > we'd also have an option to check if a user used OTP on the external > IdP or not, and require using OTP in Keycloak if not. Personally I wouldn't hardcode any OTP related options directly to IdentityProviders. Users may want something else than OTP (For example require Facebook users to reauthenticate with SMS etc). The configurable post-login flow per identity provider is more flexible. It also allows to support all the usecases above (If identityProvider itself used OTP, you can use empty post-login flow and not require users to authenticate through OTP again. Otherwise use flow with OTP required). > > In the past we had the option of requiring OTP for a realm, but that's > now gone and instead we set required on the otp authenticator in the > browser flow. However, that's no longer used in brokered logins. This > is a regression and needs fixing. Just checked how it worked in 1.2.0 when I was testing migration of something else... When OTP was in requiredCredentials for realm, then after first Facebook login we required user to setup OTP. However in all next Facebook logins of this user, we didn't require OTP login. So it's not regression. Despite first login required action, it was always like: broker OR (password AND otp). Marek > > On 23 November 2015 at 08:54, Marek Posolda > wrote: > > On 20/11/15 18:46, Dane Barentine wrote: >> I can look at the IdentityProvider Mapper and see. As you said it's a workaround though and now requires me to create a mapper and an authenticator to handle both types of authentication. >> >> Regarding this: "Isn't it the more proper option for your usecase to use OTP on the second server side instead?" >> >> In a lot of cases it would probably be more proper. But it's not always practical as we don't always control the IDPs. So the use case for OTP is if we are using an IDP that either doesn't support, or the team that is running it doesn't want to support, something like OTP but we still want to require it when logging in through Keycloak. The other use case is when we are making authentication/authorization decisions based on some other sort risk assessment or user required action. This may involve using services or code that we will never be able to plug into an IDP. In those cases a post broker login flow would allow us to add on a consistent layer across all the brokered IDPs regardless of what they can technically support. > You can use requiredAction SPI after broker authentication. That > is triggered after each authentication (classic or broker) . See > some docs here : > http://keycloak.github.io/docs/userguide/keycloak-server/html/auth_spi.html#d4e3420 > . > > Maybe using RequiredAction SPI for OTP authentication is even > better than IdentityProviderMapper (even if it's still seems to be > a workaround). In your RequiredActionProvider.evaluateTriggers(), > you will check that user was authenticated through broker and if > yes, you put some requiredAction like "authenticate_otp" to > clientSession. Then in requiredActionChallenge() you will redirect > to OTP form and in processAction() you will check if OTP was > successful. The code in requiredActionChallenge and processAction > methods might be very similar to the code in OTPFormAuthenticator. > You can likely inspire from here. > > If you still have issues, feel free to create JIRA and we will try > to look at improve things in Keycloak. > > Regards, > Marek >> Dane >> >> -----Original Message----- >> From:keycloak-dev-bounces at lists.jboss.org >> [mailto:keycloak-dev-bounces at lists.jboss.org] On Behalf Of Marek Posolda >> Sent: Friday, November 20, 2015 8:58 AM >> To: Bill Burke ;keycloak-dev at lists.jboss.org >> Subject: Re: [keycloak-dev] Identity Broker login flow >> >> Yes, however the new "First Broker Login" flow I've added is triggered just when there is no Keycloak user already linked to IdentityProvider. >> In case that there is already some Keycloak user linked to identityProvider identity, the flow is not executed. Should it be changed to trigger flow after each broker login? I didn't want to do that as there were no use-cases (until now) . And always trigger the flow might require another browser redirect.. >> >> Marek >> >> On 20/11/15 17:41, Bill Burke wrote: >>> Oh, I thought our broker flows happened after the user logged into the >>> external IDP? >>> >>> On 11/20/2015 11:18 AM, Marek Posolda wrote: >>>> Yep. I know :-\ >>>> >>>> As usually, the most proper solution requires most of the changes and >>>> refactoring... >>>> >>>> For this particular case, I wonder if Dane can use the approach with >>>> implement IdentityProviderMapper to redirect to OTP login. This is >>>> workaround though (IdentityProviderMapper is likely not designed for >>>> the usecases like this), however I think it should work. This won't >>>> require any changes in existing Keycloak code. >>>> >>>> I can introduce "post-broker-login" flow, but not sure if it's >>>> something we should support OOTB. I don't know if it's useful for >>>> more people. So I would rather skip adding post-broker-login as "authentication levels" >>>> will be more proper and more generic solution IMO. >>>> >>>> Marek >>>> >>>> >>>> On 20/11/15 16:05, Bill Burke wrote: >>>>> All something we want to do, but do we have time? Very soon we need >>>>> to start focusing on cleaning up the APIs/SPIs, documentation, >>>>> improving UI, performance, and the tons of minor problems people are >>>>> experiencing. >>>>> >>>>> On 11/20/2015 3:45 AM, Marek Posolda wrote: >>>>>> I wonder if we should address this together with reducing number of >>>>>> redirects (https://issues.jboss.org/browse/KEYCLOAK-2098 ) and >>>>>> also support for "authentication levels" . We can encapsulate all >>>>>> the state of authentication in ClientSession. If someone refreshes >>>>>> the page, we will retrieve the ClientSession from the code and the >>>>>> current state of execution will be retrieved from ClientSession itself. >>>>>> >>>>>> ClientSession will also encapsulate all the state of authentication >>>>>> after authentication is finished. For example there will be >>>>>> "cookie" if ClientSession was authenticated through cookie, >>>>>> "password, otp" if clientSession was authenticated through password >>>>>> + otp or "broker-facebook" if it was authenticated through >>>>>> Facebook. There will be also timestamps of when each successful >>>>>> authentication happened. >>>>>> The >>>>>> state will be partially saved in UserSession, so if clientSession >>>>>> was authenticated through cookie, it can retrieve from userSession >>>>>> when the "real" authentication happened etc. >>>>>> >>>>>> This will allow us to support periodic re-authentication, >>>>>> authentication levels and different requirements for >>>>>> "authentication level" for individual application. For example at >>>>>> each SSO login or refresh, we will be able to: >>>>>> - ask user for re-authenticate through OTP every 30 minutes >>>>>> - ask user for re-authenticate through OTP if he was authenticated >>>>>> through Facebook broker >>>>>> - ask user for re-authenticate through OTP if application required >>>>>> that with some magic parameter (AFAIK both OIDC or SAML has some >>>>>> parameters for specify authentication levels) >>>>>> >>>>>> Marek >>>>>> >>>>>> On 20/11/15 08:41, Marek Posolda wrote: >>>>>>> You're right, we don't have this right now. I don't know if it's >>>>>>> something we should support OOTB. The idea of broker login is, >>>>>>> that you delegate authentication to another SSO/social server. >>>>>>> Once the second server say "Ok, user is authenticated", we treat >>>>>>> him as authenticated on Keycloak side too. Isn't it the more >>>>>>> proper option for your usecase to use OTP on the second server side instead? >>>>>>> >>>>>>> Another option is to implement IdentityProviderMapper and in >>>>>>> "updateBrokeredUser" method, you will redirect user to OTP login. >>>>>>> Could you try this? >>>>>>> >>>>>>> >>>>>>> If we want to support another login flow triggered after each >>>>>>> broker login (which I am not convinced TBH), we can either: >>>>>>> >>>>>>> 1) Introduce "post-broker-login" flow, which will be configurable >>>>>>> per IdentityProvider. By default, it will be empty . >>>>>>> >>>>>>> 2) Use just one flow, which will be triggered after each broker >>>>>>> login (current "first broker login" flow is triggered only if >>>>>>> federatedIdentity doesn't yet exist in Keycloak). In this case, >>>>>>> the current "first broker login" flow will need to be renamed to >>>>>>> "broker login" and more logic will need to be moved from >>>>>>> IdentityBrokerService to the flow itself. The disadvantage of this >>>>>>> option is, that it may always require another redirect to trigger >>>>>>> authentication flows. But we're trying to reduce the number of >>>>>>> redirects ( >>>>>>> https://issues.jboss.org/browse/KEYCLOAK-2098 ) >>>>>>> >>>>>>> Marek >>>>>>> >>>>>>> On 20/11/15 00:06, Dane Barentine wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> I'm trying to add a custom authenticator and it appears that that >>>>>>>> there is no way to insert it in the flow if it's a brokered IDP >>>>>>>> login where the linked Keycloak account already exists. >>>>>>>> >>>>>>>> If it's a local Keycloak user I can use the Browser flow and if >>>>>>>> it's a new brokered user the First Broker Login flow will >>>>>>>> execute. But I don't see a flow that would allow me to insert >>>>>>>> something like OTP after a brokered login of an existing user. >>>>>>>> >>>>>>>> If I'm just missing it let me know but I think there needs to be >>>>>>>> some sort of flow for brokered logins that runs on both existing >>>>>>>> and new users. For new users it would run after the First Broker >>>>>>>> Login flow. >>>>>>>> Or better yet maybe a flow that would allow things such as OTP to >>>>>>>> happen after any brokered or local login. That way it wouldn't >>>>>>>> have to be configured in multiple flows. >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Dane >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> keycloak-dev mailing list >>>>>>>> keycloak-dev at lists.jboss.org >>>>>>>> >>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> _______________________________________________ >>>>>>> keycloak-dev mailing list >>>>>>> keycloak-dev at lists.jboss.org >>>>>>> >>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev at lists.jboss.org >>>>>> >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/0b2497b1/attachment-0001.html From mposolda at redhat.com Mon Nov 23 09:52:40 2015 From: mposolda at redhat.com (Marek Posolda) Date: Mon, 23 Nov 2015 15:52:40 +0100 Subject: [keycloak-dev] Identity Broker login flow In-Reply-To: <56531DD8.7070009@redhat.com> References: <564ECEB7.3020401@redhat.com> <564EDD95.4090101@redhat.com> <564F36CD.5000003@redhat.com> <564F47E4.3010809@redhat.com> <564F4D4A.3050805@redhat.com> <564F5113.9030202@redhat.com> <5652C63D.7070106@redhat.com> <56531DD8.7070009@redhat.com> Message-ID: <56532838.2070905@redhat.com> Created https://issues.jboss.org/browse/KEYCLOAK-2124 with fix version 1.7.0.CR1 for now. For now, I didn't assign to myself, but I hope to start on it later this week after finish implicit flow. Marek On 23/11/15 15:08, Bill Burke wrote: > We need a full post-IDP flow. Marek just added a flow that if it was > a new user. > > On 11/23/2015 4:03 AM, Stian Thorgersen wrote: >> We need to support use-cases where OTP is required after login with >> external IdP. This should be built-in IMO. We should have a easy way to >> set if OTP is required for a realm or not. Then an option on each IdP to >> configure if it can skip the OTP requirement or not. Ideally we'd also >> have an option to check if a user used OTP on the external IdP or not, >> and require using OTP in Keycloak if not. >> >> In the past we had the option of requiring OTP for a realm, but that's >> now gone and instead we set required on the otp authenticator in the >> browser flow. However, that's no longer used in brokered logins. This is >> a regression and needs fixing. >> >> On 23 November 2015 at 08:54, Marek Posolda > > wrote: >> >> On 20/11/15 18:46, Dane Barentine wrote: >>> I can look at the IdentityProvider Mapper and see. As you said >>> it's a workaround though and now requires me to create a mapper and >>> an authenticator to handle both types of authentication. >>> >>> Regarding this: "Isn't it the more proper option for your >>> usecase to use OTP on the second server side instead?" >>> >>> In a lot of cases it would probably be more proper. But it's not >>> always practical as we don't always control the IDPs. So the use >>> case for OTP is if we are using an IDP that either doesn't support, >>> or the team that is running it doesn't want to support, something >>> like OTP but we still want to require it when logging in through >>> Keycloak. The other use case is when we are making >>> authentication/authorization decisions based on some other sort risk >>> assessment or user required action. This may involve using services >>> or code that we will never be able to plug into an IDP. In those >>> cases a post broker login flow would allow us to add on a consistent >>> layer across all the brokered IDPs regardless of what they can >>> technically support. >> You can use requiredAction SPI after broker authentication. That is >> triggered after each authentication (classic or broker) . See some >> docs here : >> http://keycloak.github.io/docs/userguide/keycloak-server/html/auth_spi.html#d4e3420 >> . >> >> Maybe using RequiredAction SPI for OTP authentication is even better >> than IdentityProviderMapper (even if it's still seems to be a >> workaround). In your RequiredActionProvider.evaluateTriggers(), you >> will check that user was authenticated through broker and if yes, >> you put some requiredAction like "authenticate_otp" to >> clientSession. Then in requiredActionChallenge() you will redirect >> to OTP form and in processAction() you will check if OTP was >> successful. The code in requiredActionChallenge and processAction >> methods might be very similar to the code in OTPFormAuthenticator. >> You can likely inspire from here. >> >> If you still have issues, feel free to create JIRA and we will try >> to look at improve things in Keycloak. >> >> Regards, >> Marek >>> Dane >>> >>> -----Original Message----- >>> From:keycloak-dev-bounces at lists.jboss.org >>> >>> [mailto:keycloak-dev-bounces at lists.jboss.org] On Behalf Of Marek >>> Posolda >>> Sent: Friday, November 20, 2015 8:58 AM >>> To: Bill Burke >>> ;keycloak-dev at lists.jboss.org >>> >>> Subject: Re: [keycloak-dev] Identity Broker login flow >>> >>> Yes, however the new "First Broker Login" flow I've added is >>> triggered just when there is no Keycloak user already linked to >>> IdentityProvider. >>> In case that there is already some Keycloak user linked to >>> identityProvider identity, the flow is not executed. Should it be >>> changed to trigger flow after each broker login? I didn't want to do >>> that as there were no use-cases (until now) . And always trigger the >>> flow might require another browser redirect.. >>> >>> Marek >>> >>> On 20/11/15 17:41, Bill Burke wrote: >>>> Oh, I thought our broker flows happened after the user logged >>>> into the >>>> external IDP? >>>> >>>> On 11/20/2015 11:18 AM, Marek Posolda wrote: >>>>> Yep. I know :-\ >>>>> >>>>> As usually, the most proper solution requires most of the >>>>> changes and >>>>> refactoring... >>>>> >>>>> For this particular case, I wonder if Dane can use the >>>>> approach with >>>>> implement IdentityProviderMapper to redirect to OTP login. >>>>> This is >>>>> workarhttps://issues.jboss.org/browse/KEYCLOAK-2124ound though >>>>> (IdentityProviderMapper is likely not designed for >>>>> the usecases like this), however I think it should work. This >>>>> won't >>>>> require any changes in existing Keycloak code. >>>>> >>>>> I can introduce "post-broker-login" flow, but not sure if it's >>>>> something we should support OOTB. I don't know if it's useful for >>>>> more people. So I would rather skip adding post-broker-login >>>>> as "authentication levels" >>>>> will be more proper and more generic solution IMO. >>>>> >>>>> Marek >>>>> >>>>> >>>>> On 20/11/15 16:05, Bill Burke wrote: >>>>>> All something we want to do, but do we have time? Very soon >>>>>> we need >>>>>> to start focusing on cleaning up the APIs/SPIs, documentation, >>>>>> improving UI, performance, and the tons of minor problems >>>>>> people are >>>>>> experiencing. >>>>>> >>>>>> On 11/20/2015 3:45 AM, Marek Posolda wrote: >>>>>>> I wonder if we should address this together with reducing >>>>>>> number of >>>>>>> redirects (https://issues.jboss.org/browse/KEYCLOAK-2098 ) and >>>>>>> also support for "authentication levels" . We can >>>>>>> encapsulate all >>>>>>> the state of authentication in ClientSession. If someone >>>>>>> refreshes >>>>>>> the page, we will retrieve the ClientSession from the code >>>>>>> and the >>>>>>> current state of execution will be retrieved from >>>>>>> ClientSession itself. >>>>>>> https://issues.jboss.org/browse/KEYCLOAK-2124 >>>>>>> ClientSession will also encapsulate all the state of >>>>>>> authentication >>>>>>> after authentication is finished. For example there will be >>>>>>> "cookie" if ClientSession was authenticated through cookie, >>>>>>> "password, otp" if clientSession was authenticated through >>>>>>> password >>>>>>> + otp or "broker-facebook" if it was authenticated through >>>>>>> Facebook. There will be also timestamps of when each successful >>>>>>> authentication happened. >>>>>>> The >>>>>>> state will be partially saved in UserSession, so if >>>>>>> clientSession >>>>>>> was authenticated through cookie, it can retrieve from >>>>>>> userSession >>>>>>> when the "real" authentication happened etc. >>>>>>> >>>>>>> This will allow us to support periodic re-authentication, >>>>>>> authentication levels and different requirements for >>>>>>> "authentication level" for individual application. For >>>>>>> example at >>>>>>> each SSO login or refresh, we will be able to: >>>>>>> - ask user for re-authenticate through OTP every 30 minutes >>>>>>> - ask user for re-authenticate through OTP if he was >>>>>>> authenticated >>>>>>> through Facebook broker >>>>>>> - ask user for re-authenticate through OTP if application >>>>>>> required >>>>>>> that with some magic parameter (AFAIK both OIDC or SAML has >>>>>>> some >>>>>>> parameters for specify authentication levels) >>>>>>> >>>>>>> Marek >>>>>>> >>>>>>> On 20/11/15 08:41, Marek Posolda wrote: >>>>>>>> You're right, we don't have this right now. I don't know if >>>>>>>> it's >>>>>>>> something we should support OOTB. The idea of broker login is, >>>>>>>> that you delegate authentication to another SSO/social server. >>>>>>>> Once the second server say "Ok, user is authenticated", we >>>>>>>> treat >>>>>>>> him as authenticated on Keycloak side too. Isn't it the more >>>>>>>> proper option for your usecase to use OTP on the second >>>>>>>> server side instead? >>>>>>>> >>>>>>>> Another option is to implement IdentityProviderMapper and in >>>>>>>> "updateBrokeredUser" method, you will redirect user to OTP >>>>>>>> login. >>>>>>>> Could you try this? >>>>>>>> >>>>>>>> >>>>>>>> If we want to support another login flow triggered after each >>>>>>>> broker login (which I am not convinced TBH), we can either: >>>>>>>> >>>>>>>> 1) Introduce "post-broker-login" flow, which will be >>>>>>>> configurable >>>>>>>> per IdentityProvider. By default, it will be empty . >>>>>>>> >>>>>>>> 2) Use just one flow, which will be triggered after each >>>>>>>> broker >>>>>>>> login (current "first broker login" flow is triggered only if >>>>>>>> federatedIdentity doesn't yet exist in Keycloak). In this >>>>>>>> case, >>>>>>>> the current "first broker login" flow will need to be >>>>>>>> renamed to >>>>>>>> "broker login" and more logic will need to be moved from >>>>>>>> IdentityBrokerService to the flow itself. The disadvantage >>>>>>>> of this >>>>>>>> option is, that it may always require another redirect to >>>>>>>> trigger >>>>>>>> authentication flows. But we're trying to reduce the number of >>>>>>>> redirects ( >>>>>>>> https://issues.jboss.org/browse/KEYCLOAK-2098 ) >>>>>>>> >>>>>>>> Marek >>>>>>>> >>>>>>>> On 20/11/15 00:06, Dane Barentine wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I'm trying to add a custom authenticator and it appears >>>>>>>>> that that >>>>>>>>> there is no way to insert it in the flow if it's a >>>>>>>>> brokered IDP >>>>>>>>> login where the linked Keycloak account already exists. >>>>>>>>> >>>>>>>>> If it's a local Keycloak user I can use the Browser flow >>>>>>>>> and if >>>>>>>>> it's a new brokered user the First Broker Login flow will >>>>>>>>> execute. But I don't see a flow that would allow me to insert >>>>>>>>> something like OTP after a brokered login of an existing >>>>>>>>> user. >>>>>>>>> >>>>>>>>> If I'm just missing it let me know but I think there needs >>>>>>>>> to be >>>>>>>>> some sort of flow for brokered logins that runs on both >>>>>>>>> existing >>>>>>>>> and new users. For new users it would run after the First >>>>>>>>> Broker >>>>>>>>> Login flow. >>>>>>>>> Or better yet maybe a flow that would allow things such as >>>>>>>>> OTP to >>>>>>>>> happen after any brokered or local login. That way it >>>>>>>>> wouldn't >>>>>>>>> have to be configured in multiple flows. >>>>>>>>> >>>>>>>>> Thanhttps://issues.jboss.org/browse/KEYCLOAK-2124ks >>>>>>>>> >>>>>>>>> Dane >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> keycloak-dev mailing list >>>>>>>>> keycloak-dev at lists.jboss.org >>>>>>>>> >>>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>>> _______________________________________________ >>>>>>>> keycloak-dev mailing list >>>>>>>> keycloak-dev at lists.jboss.org >>>>>>>> >>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> _______________________________________________ >>>>>>> keycloak-dev mailing list >>>>>>> keycloak-dev at lists.jboss.org >>>>>>> >>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> > From sthorger at redhat.com Mon Nov 23 10:01:39 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 16:01:39 +0100 Subject: [keycloak-dev] Fed up with Travis Message-ID: I'm fed up with Travis. Anyone have suggestions to an alternative? My best idea now is to spin up as big an instance as we can on our public OpenStack. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/343cb99c/attachment.html From bburke at redhat.com Mon Nov 23 10:08:36 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 23 Nov 2015 10:08:36 -0500 Subject: [keycloak-dev] Fed up with Travis In-Reply-To: References: Message-ID: <56532BF4.7020005@redhat.com> Is the problem that our build takes too long? CloudBees maybe? On 11/23/2015 10:01 AM, Stian Thorgersen wrote: > I'm fed up with Travis. Anyone have suggestions to an alternative? > > My best idea now is to spin up as big an instance as we can on our > public OpenStack. > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Mon Nov 23 10:13:10 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 23 Nov 2015 10:13:10 -0500 Subject: [keycloak-dev] changing adapter error handling Message-ID: <56532D06.1070504@redhat.com> Any servlet based adapter will now be using HttpServletResponse.sendError with the appropriate Http status code of the protocol. To handle errors, users will have to add the appropriate error code handlers in web.xml. I should have done that in the first place, but quite honestly, I didn't even know that existed in the servlet spec. I'll be setting a request attribute that contains a protocol specific object so that people can view the error code, message, or anything else the protocol provides (i.e. SAML errors can be quite complex and can have their own document). Hopefully I"ll have this done by tomorrow. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Mon Nov 23 10:17:57 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 16:17:57 +0100 Subject: [keycloak-dev] Fed up with Travis In-Reply-To: <56532BF4.7020005@redhat.com> References: <56532BF4.7020005@redhat.com> Message-ID: Builds takes too long is one. Another is that the only option they have to 'cache' Maven dependencies is a tarball that's uploaded/downloaded to Amazon S3. This mechanism sometimes gets a fit and fail tests even when tests have passed. CloudBees is probably not better for speed, it's also not free. On 23 November 2015 at 16:08, Bill Burke wrote: > Is the problem that our build takes too long? CloudBees maybe? > > On 11/23/2015 10:01 AM, Stian Thorgersen wrote: > > I'm fed up with Travis. Anyone have suggestions to an alternative? > > > > My best idea now is to spin up as big an instance as we can on our > > public OpenStack. > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/3732a6d7/attachment.html From sthorger at redhat.com Mon Nov 23 10:19:02 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 16:19:02 +0100 Subject: [keycloak-dev] changing adapter error handling In-Reply-To: <56532D06.1070504@redhat.com> References: <56532D06.1070504@redhat.com> Message-ID: Are you adding documentation and example as well? On 23 November 2015 at 16:13, Bill Burke wrote: > Any servlet based adapter will now be using > HttpServletResponse.sendError with the appropriate Http status code of > the protocol. To handle errors, users will have to add the appropriate > error code handlers in web.xml. I should have done that in the first > place, but quite honestly, I didn't even know that existed in the > servlet spec. > > I'll be setting a request attribute that contains a protocol specific > object so that people can view the error code, message, or anything else > the protocol provides (i.e. SAML errors can be quite complex and can > have their own document). > > Hopefully I"ll have this done by tomorrow. > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/52e2ad29/attachment-0001.html From srossillo at smartling.com Mon Nov 23 10:19:23 2015 From: srossillo at smartling.com (Scott Rossillo) Date: Mon, 23 Nov 2015 10:19:23 -0500 Subject: [keycloak-dev] Fed up with Travis In-Reply-To: References: <56532BF4.7020005@redhat.com> Message-ID: I like CircleCI (https://circleci.com/ ). ~ Scott > On Nov 23, 2015, at 10:17 AM, Stian Thorgersen wrote: > > Builds takes too long is one. > > Another is that the only option they have to 'cache' Maven dependencies is a tarball that's uploaded/downloaded to Amazon S3. This mechanism sometimes gets a fit and fail tests even when tests have passed. > > CloudBees is probably not better for speed, it's also not free. > > On 23 November 2015 at 16:08, Bill Burke > wrote: > Is the problem that our build takes too long? CloudBees maybe? > > On 11/23/2015 10:01 AM, Stian Thorgersen wrote: > > I'm fed up with Travis. Anyone have suggestions to an alternative? > > > > My best idea now is to spin up as big an instance as we can on our > > public OpenStack. > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/2d41b2d6/attachment.html From thomas.darimont at googlemail.com Mon Nov 23 10:43:23 2015 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Mon, 23 Nov 2015 16:43:23 +0100 Subject: [keycloak-dev] Support for conditional AuthenticationFlowExecution. Message-ID: Hello group, this is my first post on this mailinglist and I want to say thank you for this awesome project! I had a look at many IDM / SSO solutions before and Keycloak provided the best out-of-the box experience so far! I posted the following in the JIRA initially but Stian Thorgersen asked me to post this on the mailing list as well. Scenario: Support for conditional AuthenticationFlowExecution. Often some authentication flow steps should only be executed under certain conditions, e.g. somtimes a TOTP based auth step is only required of requests come with a certain request header value. It would be cool if one could configure a condition on the AuthenticationFlowExecution (if I'm not mistaken) that if evaluated to true would execute or skip a particular authentication step. This could perhaps be configured via the admin console in the Authentication -> Flows tab. Conditions could perhaps be simple JavaScript expressions that could be evaluated via the built-in JavaScript ScriptEngine. For this it would be useful to provide a set of "standard" functions that can be called from the expressions (perhaps based on a whitelist). Admins should also be able to define their custom functions. The context could provide access to the current http request, current user, the requested client application and perhaps the keycloak configuration. The issue: https://issues.jboss.org/browse/KEYCLOAK-2108 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/2073c236/attachment.html From bburke at redhat.com Mon Nov 23 10:47:00 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 23 Nov 2015 10:47:00 -0500 Subject: [keycloak-dev] changing adapter error handling In-Reply-To: References: <56532D06.1070504@redhat.com> Message-ID: <565334F4.4080501@redhat.com> Yes. On 11/23/2015 10:19 AM, Stian Thorgersen wrote: > Are you adding documentation and example as well? > > On 23 November 2015 at 16:13, Bill Burke > wrote: > > Any servlet based adapter will now be using > HttpServletResponse.sendError with the appropriate Http status code of > the protocol. To handle errors, users will have to add the appropriate > error code handlers in web.xml. I should have done that in the first > place, but quite honestly, I didn't even know that existed in the > servlet spec. > > I'll be setting a request attribute that contains a protocol specific > object so that people can view the error code, message, or anything else > the protocol provides (i.e. SAML errors can be quite complex and can > have their own document). > > Hopefully I"ll have this done by tomorrow. > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Mon Nov 23 10:49:47 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 23 Nov 2015 10:49:47 -0500 Subject: [keycloak-dev] Support for conditional AuthenticationFlowExecution. In-Reply-To: References: Message-ID: <5653359B.4030308@redhat.com> "Alternative" flows/authenticators should allow you to do this. It might look/be a little awkward, but it should allow you to do conditionals. Notice that Username/Password is "Alternative". This means its not executed if Cookie is successful. You can nest this stuff too. Its not the greatest, but I wanted to avoid anything too complex that required a lot of UI work and/or some kind of scripting engine. On 11/23/2015 10:43 AM, Thomas Darimont wrote: > Hello group, > > > this is my first post on this mailinglist and I want to say thank you > for this awesome project! > > I had a look at many IDM / SSO solutions before and Keycloak provided > the best out-of-the box > > experience so far! > > > I posted the following in the JIRA initially but Stian Thorgersen asked > me to post this > > on the mailing list as well. > > > Scenario: > > Support for conditional AuthenticationFlowExecution. > > > Often some authentication flow steps should only be executed under > certain conditions, > > e.g. somtimes a TOTP based auth step is only required of requests come > with a > > certain request header value. > > It would be cool if one could configure a condition on the > AuthenticationFlowExecution > > (if I'm not mistaken) that if evaluated to true would execute or skip a > particular authentication step. > > This could perhaps be configured via the admin console in the > Authentication -> Flows tab. > > Conditions could perhaps be simple JavaScript expressions that could be > evaluated via the built-in JavaScript ScriptEngine. > > For this it would be useful to provide a set of "standard" functions > that can be called from the expressions (perhaps based on a whitelist). > > Admins should also be able to define their custom functions. > > The context could provide access to the current http request, current > user, the requested client application and perhaps the keycloak > configuration. > > > The issue: https://issues.jboss.org/browse/KEYCLOAK-2108 > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From erik.mulder at docdatapayments.com Mon Nov 23 10:54:05 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Mon, 23 Nov 2015 16:54:05 +0100 Subject: [keycloak-dev] Add your custom Spi to the Keycloak session? References: Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668B08D@DDPEX01.DDP.dcloud.local> We are extending KeyCloak with some custom functionality and we like to add an extra org.keycloak.provider.Spi and 'piggy-back' on the KeycloakSession.getProvider mechanism at various places. So we added a org.keycloak.provider.Spi file in the META-INF/services of our provider jar with the Spi implementation we made. But unfortunately, it's not picked up by the DefaultKeycloakSessionFactory, since the classloader that is used in ServiceLoader.load (DefaultKeycloakSessionFactory:56) doesn't have the custom provider jars in scope. Is this intentional? I can understand if you don't want some external Spi 'entering' the KeyCloak system. On the other hand, that's the 'own risk' of the developer creating it. We would like this constraint lifted if possible. If not, is there another way we can get our own Spi's added to the keycloak session without changing any KeyCloak sources? Like using a module instead of a 'plain' provider jar? Are these added modules in scope of the DefaultKeycloakSessionFactory classloader? From ayoung at redhat.com Mon Nov 23 11:58:43 2015 From: ayoung at redhat.com (Adam Young) Date: Mon, 23 Nov 2015 11:58:43 -0500 Subject: [keycloak-dev] Fed up with Travis In-Reply-To: References: Message-ID: <565345C3.1090806@redhat.com> On 11/23/2015 10:01 AM, Stian Thorgersen wrote: > I'm fed up with Travis. Anyone have suggestions to an alternative? > > My best idea now is to spin up as big an instance as we can on our > public OpenStack. Let me know if you go this approach. I can help. > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/e6877793/attachment-0001.html From sthorger at redhat.com Mon Nov 23 13:30:24 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 19:30:24 +0100 Subject: [keycloak-dev] Fed up with Travis In-Reply-To: <565345C3.1090806@redhat.com> References: <565345C3.1090806@redhat.com> Message-ID: On 23 November 2015 at 17:58, Adam Young wrote: > On 11/23/2015 10:01 AM, Stian Thorgersen wrote: > > I'm fed up with Travis. Anyone have suggestions to an alternative? > > My best idea now is to spin up as big an instance as we can on our public > OpenStack. > > > > Let me know if you go this approach. I can help. > Will do - I'll probably give it a go and see how long it takes to build and test on a big instance first. I imagine the instances Travis allocates are pretty small. > > > > _______________________________________________ > keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/c67793ea/attachment.html From sthorger at redhat.com Mon Nov 23 14:04:59 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 23 Nov 2015 20:04:59 +0100 Subject: [keycloak-dev] Add your custom Spi to the Keycloak session? In-Reply-To: <9A5619B792BBA041AE094585791BB71C0137B668B08D@DDPEX01.DDP.dcloud.local> References: <9A5619B792BBA041AE094585791BB71C0137B668B08D@DDPEX01.DDP.dcloud.local> Message-ID: No specific reason it can't. Feel free to create a JIRA to request. It's not something we can prioritize right now though. If you're willing to do a PR I can give you some pointers on how to implement it. On 23 November 2015 at 16:54, Erik Mulder wrote: > We are extending KeyCloak with some custom functionality and we like to > add an extra org.keycloak.provider.Spi and 'piggy-back' on the > KeycloakSession.getProvider mechanism at various places. So we added a > org.keycloak.provider.Spi file in the META-INF/services of our provider > jar with the Spi implementation we made. But unfortunately, it's not > picked up by the DefaultKeycloakSessionFactory, since the classloader > that is used in ServiceLoader.load (DefaultKeycloakSessionFactory:56) > doesn't have the custom provider jars in scope. Is this intentional? > > I can understand if you don't want some external Spi 'entering' the > KeyCloak system. On the other hand, that's the 'own risk' of the > developer creating it. We would like this constraint lifted if possible. > If not, is there another way we can get our own Spi's added to the > keycloak session without changing any KeyCloak sources? Like using a > module instead of a 'plain' provider jar? Are these added modules in > scope of the DefaultKeycloakSessionFactory classloader? > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151123/904a96b9/attachment.html From erik.mulder at docdatapayments.com Mon Nov 23 16:13:36 2015 From: erik.mulder at docdatapayments.com (Erik Mulder) Date: Mon, 23 Nov 2015 22:13:36 +0100 Subject: [keycloak-dev] Add your custom Spi to the Keycloak session? In-Reply-To: References: <9A5619B792BBA041AE094585791BB71C0137B668B08D@DDPEX01.DDP.dcloud.local>, Message-ID: <9A5619B792BBA041AE094585791BB71C0137B668FD4E@DDPEX01.DDP.dcloud.local> I'll be happy to create a PR for this if you can provide me some info & tips on how to get this working. ________________________________________ Van: Stian Thorgersen [sthorger at redhat.com] Verzonden: maandag 23 november 2015 20:04 Aan: Erik Mulder CC: keycloak-dev Onderwerp: Re: [keycloak-dev] Add your custom Spi to the Keycloak session? No specific reason it can't. Feel free to create a JIRA to request. It's not something we can prioritize right now though. If you're willing to do a PR I can give you some pointers on how to implement it. On 23 November 2015 at 16:54, Erik Mulder > wrote: We are extending KeyCloak with some custom functionality and we like to add an extra org.keycloak.provider.Spi and 'piggy-back' on the KeycloakSession.getProvider mechanism at various places. So we added a org.keycloak.provider.Spi file in the META-INF/services of our provider jar with the Spi implementation we made. But unfortunately, it's not picked up by the DefaultKeycloakSessionFactory, since the classloader that is used in ServiceLoader.load (DefaultKeycloakSessionFactory:56) doesn't have the custom provider jars in scope. Is this intentional? I can understand if you don't want some external Spi 'entering' the KeyCloak system. On the other hand, that's the 'own risk' of the developer creating it. We would like this constraint lifted if possible. If not, is there another way we can get our own Spi's added to the keycloak session without changing any KeyCloak sources? Like using a module instead of a 'plain' provider jar? Are these added modules in scope of the DefaultKeycloakSessionFactory classloader? _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev From sthorger at redhat.com Tue Nov 24 03:03:08 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 24 Nov 2015 09:03:08 +0100 Subject: [keycloak-dev] Add your custom Spi to the Keycloak session? In-Reply-To: <9A5619B792BBA041AE094585791BB71C0137B668FD4E@DDPEX01.DDP.dcloud.local> References: <9A5619B792BBA041AE094585791BB71C0137B668B08D@DDPEX01.DDP.dcloud.local> <9A5619B792BBA041AE094585791BB71C0137B668FD4E@DDPEX01.DDP.dcloud.local> Message-ID: ProviderLoader is responsible to load providers of an SPI. There's only one implementations of it, this is DefaultProviderLoader. There's 3 different factories that use, which loads it differently, but you shouldn't need to worry about those. Add "List loadSPI" to ProviderLoader and implement it in DefaultProviderLoader. Implementation should be simple, use ServiceLoader and the attached classloader to find instances of SPI. Next add same method to ProviderManager. This should loop through all registered ProviderLoaders and retrieve SPIs. Final step in DefaultKeycloakSessionFactory.init use ProviderManager#loadSPI instead of ServiceLoader to list SPIs. You also need to document it under the provider section in the documentation. One comment should be that SPIs have to be deployed as modules, not just dropped into providers directory. On 23 November 2015 at 22:13, Erik Mulder wrote: > I'll be happy to create a PR for this if you can provide me some info & > tips on how to get this working. > > ________________________________________ > Van: Stian Thorgersen [sthorger at redhat.com] > Verzonden: maandag 23 november 2015 20:04 > Aan: Erik Mulder > CC: keycloak-dev > Onderwerp: Re: [keycloak-dev] Add your custom Spi to the Keycloak session? > > No specific reason it can't. Feel free to create a JIRA to request. > > It's not something we can prioritize right now though. If you're willing > to do a PR I can give you some pointers on how to implement it. > > On 23 November 2015 at 16:54, Erik Mulder > wrote: > We are extending KeyCloak with some custom functionality and we like to > add an extra org.keycloak.provider.Spi and 'piggy-back' on the > KeycloakSession.getProvider mechanism at various places. So we added a > org.keycloak.provider.Spi file in the META-INF/services of our provider > jar with the Spi implementation we made. But unfortunately, it's not > picked up by the DefaultKeycloakSessionFactory, since the classloader > that is used in ServiceLoader.load (DefaultKeycloakSessionFactory:56) > doesn't have the custom provider jars in scope. Is this intentional? > > I can understand if you don't want some external Spi 'entering' the > KeyCloak system. On the other hand, that's the 'own risk' of the > developer creating it. We would like this constraint lifted if possible. > If not, is there another way we can get our own Spi's added to the > keycloak session without changing any KeyCloak sources? Like using a > module instead of a 'plain' provider jar? Are these added modules in > scope of the DefaultKeycloakSessionFactory classloader? > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/a2e1a7d1/attachment.html From thomas.darimont at googlemail.com Tue Nov 24 08:05:13 2015 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Tue, 24 Nov 2015 14:05:13 +0100 Subject: [keycloak-dev] Keycloak Issue Board in JIRA Message-ID: Hi, the Keycloak issue overview board [1] with the 3+ lanes (Todo, In-Progress, Done) isn't (publicly?) available anymore since a few days. Is there any chance to get it back? It was very helpful to keep track about the project progress / issues. Cheers, Thomas [1] https://issues.jboss.org/projects/KEYCLOAK -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/927e0cf7/attachment.html From sthorger at redhat.com Tue Nov 24 08:09:21 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 24 Nov 2015 14:09:21 +0100 Subject: [keycloak-dev] Keycloak Issue Board in JIRA In-Reply-To: References: Message-ID: No, we don't use it ourselves so we removed id. It was a mistake that it was there in the first place. A better way to track what's going on is to look at issues assigned to the next release (https://issues.jboss.org/browse/KEYCLOAK-2124?filter=12322375 ). On 24 November 2015 at 14:05, Thomas Darimont < thomas.darimont at googlemail.com> wrote: > Hi, > > the Keycloak issue overview board [1] with the 3+ lanes (Todo, > In-Progress, Done) isn't > (publicly?) available anymore since a few days. > Is there any chance to get it back? It was very helpful to keep track > about the project progress / issues. > > Cheers, > Thomas > > [1] https://issues.jboss.org/projects/KEYCLOAK > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/b8dca96d/attachment-0001.html From mposolda at redhat.com Tue Nov 24 08:14:49 2015 From: mposolda at redhat.com (Marek Posolda) Date: Tue, 24 Nov 2015 14:14:49 +0100 Subject: [keycloak-dev] Issues with Travis In-Reply-To: References: Message-ID: <565462C9.2010004@redhat.com> Not sure it's related, but just noticed one interesting issue. When base testsuite ( testsuite/integration ) fails with error, the travis still continue build with distribution and testsuite-arquillian . This doesn't seem to be correct. IMO we should rather abort the build and not continue with testsuite-arquillian if earlier phase failed. Because in case that testsuite-arquillian pass, even if testsuite/integration failed in earlier phase, you may not notice that error happened earlier. For example see this build: https://s3.amazonaws.com/archive.travis-ci.org/jobs/92911215/log.txt Marek On 23/11/15 15:36, Stian Thorgersen wrote: > There's some issues with Travis and it keeps failing PRs even though > the tests pass. I'm looking at it and will update once it's resolved. > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/18974d4a/attachment.html From thomas.darimont at googlemail.com Tue Nov 24 08:18:09 2015 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Tue, 24 Nov 2015 14:18:09 +0100 Subject: [keycloak-dev] Keycloak Issue Board in JIRA In-Reply-To: References: Message-ID: Thanks for the quick response, that's sad to hear - I used it quite intensely to find potential issues / candidates for fixes / pr's at a glance. I'll adapt my workflow then... Cheers, Thomas 2015-11-24 14:09 GMT+01:00 Stian Thorgersen : > No, we don't use it ourselves so we removed id. It was a mistake that it > was there in the first place. > > A better way to track what's going on is to look at issues assigned to the > next release ( > https://issues.jboss.org/browse/KEYCLOAK-2124?filter=12322375). > > > > On 24 November 2015 at 14:05, Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > >> Hi, >> >> the Keycloak issue overview board [1] with the 3+ lanes (Todo, >> In-Progress, Done) isn't >> (publicly?) available anymore since a few days. >> Is there any chance to get it back? It was very helpful to keep track >> about the project progress / issues. >> >> Cheers, >> Thomas >> >> [1] https://issues.jboss.org/projects/KEYCLOAK >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/57474daf/attachment.html From thomas.darimont at googlemail.com Tue Nov 24 09:09:53 2015 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Tue, 24 Nov 2015 15:09:53 +0100 Subject: [keycloak-dev] Keycloak Issue Board in JIRA In-Reply-To: References: Message-ID: It seems that some other functionality was disabled as well - e.g. I cannot add links to PR's to issues anymore, was this intended as well or is this a mistake? 2015-11-24 14:18 GMT+01:00 Thomas Darimont : > Thanks for the quick response, > > that's sad to hear - I used it quite intensely to find potential issues / > candidates for fixes / pr's at a glance. > > I'll adapt my workflow then... > > Cheers, > Thomas > > 2015-11-24 14:09 GMT+01:00 Stian Thorgersen : > >> No, we don't use it ourselves so we removed id. It was a mistake that it >> was there in the first place. >> >> A better way to track what's going on is to look at issues assigned to >> the next release ( >> https://issues.jboss.org/browse/KEYCLOAK-2124?filter=12322375). >> >> >> >> On 24 November 2015 at 14:05, Thomas Darimont < >> thomas.darimont at googlemail.com> wrote: >> >>> Hi, >>> >>> the Keycloak issue overview board [1] with the 3+ lanes (Todo, >>> In-Progress, Done) isn't >>> (publicly?) available anymore since a few days. >>> Is there any chance to get it back? It was very helpful to keep track >>> about the project progress / issues. >>> >>> Cheers, >>> Thomas >>> >>> [1] https://issues.jboss.org/projects/KEYCLOAK >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/ba604ca7/attachment.html From sthorger at redhat.com Tue Nov 24 09:21:13 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 24 Nov 2015 15:21:13 +0100 Subject: [keycloak-dev] Keycloak Issue Board in JIRA In-Reply-To: References: Message-ID: There's been no changes to that. Only change was that the unused Kanban boards where deleted. When you say you can't add links wdym? It's working fine here. On 24 November 2015 at 15:09, Thomas Darimont < thomas.darimont at googlemail.com> wrote: > It seems that some other functionality was disabled as well - e.g. I > cannot add links to PR's to issues anymore, was this intended as well or is > this a mistake? > > 2015-11-24 14:18 GMT+01:00 Thomas Darimont >: > >> Thanks for the quick response, >> >> that's sad to hear - I used it quite intensely to find potential issues / >> candidates for fixes / pr's at a glance. >> >> I'll adapt my workflow then... >> >> Cheers, >> Thomas >> >> 2015-11-24 14:09 GMT+01:00 Stian Thorgersen : >> >>> No, we don't use it ourselves so we removed id. It was a mistake that it >>> was there in the first place. >>> >>> A better way to track what's going on is to look at issues assigned to >>> the next release ( >>> https://issues.jboss.org/browse/KEYCLOAK-2124?filter=12322375). >>> >>> >>> >>> On 24 November 2015 at 14:05, Thomas Darimont < >>> thomas.darimont at googlemail.com> wrote: >>> >>>> Hi, >>>> >>>> the Keycloak issue overview board [1] with the 3+ lanes (Todo, >>>> In-Progress, Done) isn't >>>> (publicly?) available anymore since a few days. >>>> Is there any chance to get it back? It was very helpful to keep track >>>> about the project progress / issues. >>>> >>>> Cheers, >>>> Thomas >>>> >>>> [1] https://issues.jboss.org/projects/KEYCLOAK >>>> >>>> _______________________________________________ >>>> keycloak-dev mailing list >>>> keycloak-dev at lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/ef5aba27/attachment.html From thomas.darimont at googlemail.com Tue Nov 24 09:42:17 2015 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Tue, 24 Nov 2015 15:42:17 +0100 Subject: [keycloak-dev] Keycloak Issue Board in JIRA In-Reply-To: References: Message-ID: I meant that I cannot attach a Git Pull Request link to an issue (as an authenticated user) - this was possible until few days ago. 2015-11-24 15:21 GMT+01:00 Stian Thorgersen : > There's been no changes to that. Only change was that the unused Kanban > boards where deleted. > > When you say you can't add links wdym? It's working fine here. > > On 24 November 2015 at 15:09, Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > >> It seems that some other functionality was disabled as well - e.g. I >> cannot add links to PR's to issues anymore, was this intended as well or is >> this a mistake? >> >> 2015-11-24 14:18 GMT+01:00 Thomas Darimont < >> thomas.darimont at googlemail.com>: >> >>> Thanks for the quick response, >>> >>> that's sad to hear - I used it quite intensely to find potential issues >>> / candidates for fixes / pr's at a glance. >>> >>> I'll adapt my workflow then... >>> >>> Cheers, >>> Thomas >>> >>> 2015-11-24 14:09 GMT+01:00 Stian Thorgersen : >>> >>>> No, we don't use it ourselves so we removed id. It was a mistake that >>>> it was there in the first place. >>>> >>>> A better way to track what's going on is to look at issues assigned to >>>> the next release ( >>>> https://issues.jboss.org/browse/KEYCLOAK-2124?filter=12322375). >>>> >>>> >>>> >>>> On 24 November 2015 at 14:05, Thomas Darimont < >>>> thomas.darimont at googlemail.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> the Keycloak issue overview board [1] with the 3+ lanes (Todo, >>>>> In-Progress, Done) isn't >>>>> (publicly?) available anymore since a few days. >>>>> Is there any chance to get it back? It was very helpful to keep track >>>>> about the project progress / issues. >>>>> >>>>> Cheers, >>>>> Thomas >>>>> >>>>> [1] https://issues.jboss.org/projects/KEYCLOAK >>>>> >>>>> _______________________________________________ >>>>> keycloak-dev mailing list >>>>> keycloak-dev at lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/0d4bdc86/attachment-0001.html From sthorger at redhat.com Tue Nov 24 09:55:51 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Tue, 24 Nov 2015 15:55:51 +0100 Subject: [keycloak-dev] Keycloak Issue Board in JIRA In-Reply-To: References: Message-ID: I understood that, but can you describe why it's not possible? It works here and nothing should have changed so more info would help to figure out what's going on. On 24 November 2015 at 15:42, Thomas Darimont < thomas.darimont at googlemail.com> wrote: > I meant that I cannot attach a Git Pull Request link to an issue (as an > authenticated user) - this was possible until few days ago. > > 2015-11-24 15:21 GMT+01:00 Stian Thorgersen : > >> There's been no changes to that. Only change was that the unused Kanban >> boards where deleted. >> >> When you say you can't add links wdym? It's working fine here. >> >> On 24 November 2015 at 15:09, Thomas Darimont < >> thomas.darimont at googlemail.com> wrote: >> >>> It seems that some other functionality was disabled as well - e.g. I >>> cannot add links to PR's to issues anymore, was this intended as well or is >>> this a mistake? >>> >>> 2015-11-24 14:18 GMT+01:00 Thomas Darimont < >>> thomas.darimont at googlemail.com>: >>> >>>> Thanks for the quick response, >>>> >>>> that's sad to hear - I used it quite intensely to find potential issues >>>> / candidates for fixes / pr's at a glance. >>>> >>>> I'll adapt my workflow then... >>>> >>>> Cheers, >>>> Thomas >>>> >>>> 2015-11-24 14:09 GMT+01:00 Stian Thorgersen : >>>> >>>>> No, we don't use it ourselves so we removed id. It was a mistake that >>>>> it was there in the first place. >>>>> >>>>> A better way to track what's going on is to look at issues assigned to >>>>> the next release ( >>>>> https://issues.jboss.org/browse/KEYCLOAK-2124?filter=12322375). >>>>> >>>>> >>>>> >>>>> On 24 November 2015 at 14:05, Thomas Darimont < >>>>> thomas.darimont at googlemail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> the Keycloak issue overview board [1] with the 3+ lanes (Todo, >>>>>> In-Progress, Done) isn't >>>>>> (publicly?) available anymore since a few days. >>>>>> Is there any chance to get it back? It was very helpful to keep track >>>>>> about the project progress / issues. >>>>>> >>>>>> Cheers, >>>>>> Thomas >>>>>> >>>>>> [1] https://issues.jboss.org/projects/KEYCLOAK >>>>>> >>>>>> _______________________________________________ >>>>>> keycloak-dev mailing list >>>>>> keycloak-dev at lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>> >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/9f593088/attachment.html From thomas.darimont at googlemail.com Tue Nov 24 10:00:53 2015 From: thomas.darimont at googlemail.com (Thomas Darimont) Date: Tue, 24 Nov 2015 16:00:53 +0100 Subject: [keycloak-dev] Keycloak Issue Board in JIRA In-Reply-To: References: Message-ID: Well, up until a few days ago I could browse to an issue -> press "." (to bring up the jira "quick-actions" menu) -> type in "git" -> and then I could see a quick action to link a git(hub) pull request. A longer way (IIRC) was via Link -> Add Link -> Git Pull Request (which is now gone as well). HTH, Cheers Thomas 2015-11-24 15:55 GMT+01:00 Stian Thorgersen : > I understood that, but can you describe why it's not possible? It works > here and nothing should have changed so more info would help to figure out > what's going on. > > On 24 November 2015 at 15:42, Thomas Darimont < > thomas.darimont at googlemail.com> wrote: > >> I meant that I cannot attach a Git Pull Request link to an issue (as an >> authenticated user) - this was possible until few days ago. >> >> 2015-11-24 15:21 GMT+01:00 Stian Thorgersen : >> >>> There's been no changes to that. Only change was that the unused Kanban >>> boards where deleted. >>> >>> When you say you can't add links wdym? It's working fine here. >>> >>> On 24 November 2015 at 15:09, Thomas Darimont < >>> thomas.darimont at googlemail.com> wrote: >>> >>>> It seems that some other functionality was disabled as well - e.g. I >>>> cannot add links to PR's to issues anymore, was this intended as well or is >>>> this a mistake? >>>> >>>> 2015-11-24 14:18 GMT+01:00 Thomas Darimont < >>>> thomas.darimont at googlemail.com>: >>>> >>>>> Thanks for the quick response, >>>>> >>>>> that's sad to hear - I used it quite intensely to find potential >>>>> issues / candidates for fixes / pr's at a glance. >>>>> >>>>> I'll adapt my workflow then... >>>>> >>>>> Cheers, >>>>> Thomas >>>>> >>>>> 2015-11-24 14:09 GMT+01:00 Stian Thorgersen : >>>>> >>>>>> No, we don't use it ourselves so we removed id. It was a mistake that >>>>>> it was there in the first place. >>>>>> >>>>>> A better way to track what's going on is to look at issues assigned >>>>>> to the next release ( >>>>>> https://issues.jboss.org/browse/KEYCLOAK-2124?filter=12322375). >>>>>> >>>>>> >>>>>> >>>>>> On 24 November 2015 at 14:05, Thomas Darimont < >>>>>> thomas.darimont at googlemail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> the Keycloak issue overview board [1] with the 3+ lanes (Todo, >>>>>>> In-Progress, Done) isn't >>>>>>> (publicly?) available anymore since a few days. >>>>>>> Is there any chance to get it back? It was very helpful to keep >>>>>>> track about the project progress / issues. >>>>>>> >>>>>>> Cheers, >>>>>>> Thomas >>>>>>> >>>>>>> [1] https://issues.jboss.org/projects/KEYCLOAK >>>>>>> >>>>>>> _______________________________________________ >>>>>>> keycloak-dev mailing list >>>>>>> keycloak-dev at lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151124/12c22831/attachment.html From bburke at redhat.com Wed Nov 25 09:27:07 2015 From: bburke at redhat.com (Bill Burke) Date: Wed, 25 Nov 2015 09:27:07 -0500 Subject: [keycloak-dev] build is really really slow Message-ID: <5655C53B.8050208@redhat.com> Did somebody introduce something weird? Build on my machine is really really slow now. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Wed Nov 25 09:51:43 2015 From: bburke at redhat.com (Bill Burke) Date: Wed, 25 Nov 2015 09:51:43 -0500 Subject: [keycloak-dev] build is really really slow In-Reply-To: <5655C53B.8050208@redhat.com> References: <5655C53B.8050208@redhat.com> Message-ID: <5655CAFF.8070407@redhat.com> doubling permgen/heap to 2g and 4g respectively helped a little. On 11/25/2015 9:27 AM, Bill Burke wrote: > Did somebody introduce something weird? Build on my machine is really > really slow now. > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Wed Nov 25 10:00:58 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Wed, 25 Nov 2015 16:00:58 +0100 Subject: [keycloak-dev] build is really really slow In-Reply-To: <5655CAFF.8070407@redhat.com> References: <5655C53B.8050208@redhat.com> <5655CAFF.8070407@redhat.com> Message-ID: Haven't noticed any increase in build time here. Nor on Travis. Also, I'm just using "-Xms512m -Xmx1024m" for mvn. On 25 November 2015 at 15:51, Bill Burke wrote: > doubling permgen/heap to 2g and 4g respectively helped a little. > > On 11/25/2015 9:27 AM, Bill Burke wrote: > > Did somebody introduce something weird? Build on my machine is really > > really slow now. > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151125/89d719a2/attachment-0001.html From ayoung at redhat.com Wed Nov 25 12:01:07 2015 From: ayoung at redhat.com (Adam Young) Date: Wed, 25 Nov 2015 12:01:07 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <56532572.5050801@redhat.com> References: <5652763D.3050304@redhat.com> <56528D11.3010308@redhat.com> <5653212D.8010706@redhat.com> <56532572.5050801@redhat.com> Message-ID: <5655E953.3030507@redhat.com> On 11/23/2015 09:40 AM, Bill Burke wrote: > > On 11/23/2015 9:22 AM, Adam Young wrote: >> On 11/22/2015 10:50 PM, Bill Burke wrote: >>> Most Java EE apps, simple permission stuff is handled by the >>> applications themselves. The security layer propagates identity and >>> user/role mappings and the application decides which roles have >>> permission to access which resource. So, Keycloak core can focus on >>> provide user/group/role/attribute authentication and information. The >>> service Pedro is working on will offer an optional, complimentary way to >>> centrally manage permissions if you want to. My on personal opinion is >>> that most apps will want to manage resource permissions in their own >>> special app-specific way and Pedro's stuff will be most useful as a >>> backend-service. >> So the analogue in Keystone is the Policy stuff. Again, Keystone has >> had to weather a storm; >> >> Access Control permissions in OpenStack are done at the Python API (not >> URL) level. The code calls out to the policy API to check that the >> context of the call matches the attributes of the resource. Practically >> speaking, this means two checks: does the role match, and does the >> scope (project or user) match. >> > We really should have a BJ session. Basic Java EE does permission via > URL. Pedro's stuff will be a lot more abstract. You would submit your > parameters (user, group, role, resource attributes, etc.) and get back > authorization permissions. All through the UMA protocol. URL is just > one type of resource in Pedro's stuff...Pedro should answer though. > Stian and I really only have a high level understanding of what he is doing. > >> It would be easier for deployers if it was URL based. Instead, it is >> deep in the code, and the reason being that often an application needs >> to fetch an object from the database to check ownership before checking >> scope (what project owns the resource). What we learned is that this >> can and should actually be two checks. The role check can be done in a >> non-application specific way, and performed in the middleware stack; > We don't know shit about the Python stack. Personally I haven't touched > Python in 15 years. But anything Java based we have a good pulse on. > We've focused mostly on basic Java EE authorization. Most of our users > are interested in Keycloak out of the box, or customizing the federation > and/or authentication flows. This is really not a Python issue, rather it is how OpenStack has evolved. People forked off the main project and cargo-culted in the Access control implementations: good, bad and Ugly. But the lesson learned is that there are two layers: URL based and underlying resource. You can't count on the URL layer to properly protect access at the persistence layer. You saw this reflected in J2EE way back with Servlet VS Entity-EJB protections, and the modern evolution of them as well. JEE/JPA seems to have a "Role" check as its basic level of control. What we've seen is this is only half of what we need. THe role needs to be assigned on some collective aspect of ownership in order for the pattern to scale. Lets say that we are managing virtual machines. There are three levels of VMs: development, qe, production. If you say that anyone withthe "VM_MANAGER" role can create a VM, they will be able to create VMs for dev, qe, and production. This is not what you want. Instead: A Dev should be able to create a VM in Dev, view it in QE, not be to do anything in production A QE-Engineer should be able to create a VM in QE, view it in Dev, and not do anything in production Production Level VMs can only be managed by automated processes that are kicked off based on a quorum (at least 2 votes) of QE engineers. There are multiple Dev groups: AccountsReceivable, AccountsPayable, GeneraLedger, Tax. Each can work within their own set of VMs only, but all must have the same set of rules. Within a development group, there are a handful of roles: developers, DBa, Manager, QA (assigned over from the core QA org. These people can do various things to the VMs inside their own projects, but nothing inside the other dev groups. You get the idea. The roles and access permissions need to be scalable. It can't be one size fits all, and it can't be hardcoded. The general mechanism is a tuple: role, actor, target. Role is a string that describes what the user can do. Actor is User or Group of users Target is the container for the resources. In OpenStack, we have two: Projects and Domains, but this is a mistake, as we could have used Project for both. If you were managing Wordpress deployments, the roles would be: Administrator Editor Subscriber Author Contributor And the Target would be the actual instance: and editor on one may be an author on another. The real power of this model comes in delegation. You want to know who assigned what role to whom. In OpenStack, we are working toward a unified delegation model, that links short term delegations (Trusts and OAuth) with Role Assignments. This is up for review here: https://review.openstack.org/#/c/189816/15 There is a real need to be able to assign identities to non-human entities, and to be able to delegate the appropriate permissions to them. In FreeIPA, we are making more and more use of the Service principals to do that. > > >> kinda like in the Tomcat Webserver prior to hitting the servlet itself, >> but in a python WSGI way instead. This part could and should be >> dynamically fetched from the Keystone server. Only the scope check >> needs to be done in a resource specific way. There is support at the >> EJB/Hibernate layer for this kind of access control, too, and that may >> be a better place to focus on completely. I suspect it is this kind of >> "Dynamic Policy" stuff that Pedro is working on. Do we have docs for >> that? I'd love to compare notes. >> >> I'm in the US, on East Coast time. THis is a short Week due to >> Thanksgiving, but I'd love to have a video conference of some sort the >> following week to sync up the lessons learned from both projects. > Yes! Let's figure something out. > > > From bburke at redhat.com Wed Nov 25 12:10:05 2015 From: bburke at redhat.com (Bill Burke) Date: Wed, 25 Nov 2015 12:10:05 -0500 Subject: [keycloak-dev] user groups vs. client groups In-Reply-To: <5655E953.3030507@redhat.com> References: <5652763D.3050304@redhat.com> <56528D11.3010308@redhat.com> <5653212D.8010706@redhat.com> <56532572.5050801@redhat.com> <5655E953.3030507@redhat.com> Message-ID: <5655EB6D.4080904@redhat.com> I only speed read your response (I'll review again after thanksgiving), but I believe Hawkular has similar role/permission requirements. As I said before, Keycloak is currently weak on authorization. A User has Group Membership and Role mappings and attributes. That's it right now. How the application applies that security metadata is up to the app. Excited to talk to you though when Pedro gets back from vacation. On 11/25/2015 12:01 PM, Adam Young wrote: > On 11/23/2015 09:40 AM, Bill Burke wrote: >> >> On 11/23/2015 9:22 AM, Adam Young wrote: >>> On 11/22/2015 10:50 PM, Bill Burke wrote: >>>> Most Java EE apps, simple permission stuff is handled by the >>>> applications themselves. The security layer propagates identity and >>>> user/role mappings and the application decides which roles have >>>> permission to access which resource. So, Keycloak core can focus on >>>> provide user/group/role/attribute authentication and information. The >>>> service Pedro is working on will offer an optional, complimentary way to >>>> centrally manage permissions if you want to. My on personal opinion is >>>> that most apps will want to manage resource permissions in their own >>>> special app-specific way and Pedro's stuff will be most useful as a >>>> backend-service. >>> So the analogue in Keystone is the Policy stuff. Again, Keystone has >>> had to weather a storm; >>> >>> Access Control permissions in OpenStack are done at the Python API (not >>> URL) level. The code calls out to the policy API to check that the >>> context of the call matches the attributes of the resource. Practically >>> speaking, this means two checks: does the role match, and does the >>> scope (project or user) match. >>> >> We really should have a BJ session. Basic Java EE does permission via >> URL. Pedro's stuff will be a lot more abstract. You would submit your >> parameters (user, group, role, resource attributes, etc.) and get back >> authorization permissions. All through the UMA protocol. URL is just >> one type of resource in Pedro's stuff...Pedro should answer though. >> Stian and I really only have a high level understanding of what he is doing. >> >>> It would be easier for deployers if it was URL based. Instead, it is >>> deep in the code, and the reason being that often an application needs >>> to fetch an object from the database to check ownership before checking >>> scope (what project owns the resource). What we learned is that this >>> can and should actually be two checks. The role check can be done in a >>> non-application specific way, and performed in the middleware stack; >> We don't know shit about the Python stack. Personally I haven't touched >> Python in 15 years. But anything Java based we have a good pulse on. >> We've focused mostly on basic Java EE authorization. Most of our users >> are interested in Keycloak out of the box, or customizing the federation >> and/or authentication flows. > > This is really not a Python issue, rather it is how OpenStack has > evolved. People forked off the main project and cargo-culted in the > Access control implementations: good, bad and Ugly. > > But the lesson learned is that there are two layers: URL based and > underlying resource. You can't count on the URL layer to properly > protect access at the persistence layer. You saw this reflected in J2EE > way back with Servlet VS Entity-EJB protections, and the modern > evolution of them as well. > > JEE/JPA seems to have a "Role" check as its basic level of control. What > we've seen is this is only half of what we need. THe role needs to be > assigned on some collective aspect of ownership in order for the pattern > to scale. > > > Lets say that we are managing virtual machines. There are three levels > of VMs: development, qe, production. > > > If you say that anyone withthe "VM_MANAGER" role can create a VM, they > will be able to create VMs for dev, qe, and production. This is not > what you want. Instead: > > > A Dev should be able to create a VM in Dev, view it in QE, not be to do > anything in production > > A QE-Engineer should be able to create a VM in QE, view it in Dev, and > not do anything in production > > Production Level VMs can only be managed by automated processes that are > kicked off based on a quorum (at least 2 votes) of QE engineers. > > There are multiple Dev groups: AccountsReceivable, AccountsPayable, > GeneraLedger, Tax. Each can work within their own set of VMs only, but > all must have the same set of rules. > > > Within a development group, there are a handful of roles: developers, > DBa, Manager, QA (assigned over from the core QA org. These people can > do various things to the VMs inside their own projects, but nothing > inside the other dev groups. > > > > You get the idea. The roles and access permissions need to be > scalable. It can't be one size fits all, and it can't be hardcoded. > > > The general mechanism is a tuple: > > > role, actor, target. > > > Role is a string that describes what the user can do. > Actor is User or Group of users > Target is the container for the resources. In OpenStack, we have two: > Projects and Domains, but this is a mistake, as we could have used > Project for both. > > > If you were managing Wordpress deployments, the roles would be: > > Administrator > Editor > Subscriber > Author > Contributor > > And the Target would be the actual instance: and editor on one may be > an author on another. > > The real power of this model comes in delegation. You want to know who > assigned what role to whom. > > In OpenStack, we are working toward a unified delegation model, that > links short term delegations (Trusts and OAuth) with Role Assignments. > > This is up for review here: > https://review.openstack.org/#/c/189816/15 > > There is a real need to be able to assign identities to non-human > entities, and to be able to delegate the appropriate permissions to > them. In FreeIPA, we are making more and more use of the Service > principals to do that. > > > >> >> >>> kinda like in the Tomcat Webserver prior to hitting the servlet itself, >>> but in a python WSGI way instead. This part could and should be >>> dynamically fetched from the Keystone server. Only the scope check >>> needs to be done in a resource specific way. There is support at the >>> EJB/Hibernate layer for this kind of access control, too, and that may >>> be a better place to focus on completely. I suspect it is this kind of >>> "Dynamic Policy" stuff that Pedro is working on. Do we have docs for >>> that? I'd love to compare notes. >>> >>> I'm in the US, on East Coast time. THis is a short Week due to >>> Thanksgiving, but I'd love to have a video conference of some sort the >>> following week to sync up the lessons learned from both projects. >> Yes! Let's figure something out. >> >> >> > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From althaf1088 at gmail.com Thu Nov 26 01:24:31 2015 From: althaf1088 at gmail.com (mohammed althaf) Date: Thu, 26 Nov 2015 14:24:31 +0800 Subject: [keycloak-dev] Keycloak in Tomcat server Message-ID: Hi Team, I was able to use keycloak for sso seamlessly in jboss /wildfly server. Since my web application is running in tomcat ,i am trying to deploy the keycloak server as well in Tomcat, I followed https://reachmnadeem.wordpress.com/2015/01/14/deploying-keycloak-in-tomcat/ and i was able to spin up the keycloak in tomcat server.But it was using the 1.0.2 version. So i am trying to deploy the keycloak server 1.6.1.I faced the below issues. 1.Failed to provide mem for realmcache(please check failed to find provider log) So i removed the cache from keycloak-server.json Not sure it is fine?? 2 .Then i was able to procced but it again failed with h2.db exception(please check lockout.log) It will be great if you guys can help me resolve this issue. Thanks, Althaf -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151126/679270b4/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: Failed to find provider mem for realmCache.log Type: application/octet-stream Size: 7812 bytes Desc: not available Url : http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151126/679270b4/attachment-0002.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: lockout.log Type: application/octet-stream Size: 16778 bytes Desc: not available Url : http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151126/679270b4/attachment-0003.obj From sthorger at redhat.com Thu Nov 26 02:40:08 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 26 Nov 2015 08:40:08 +0100 Subject: [keycloak-dev] Keycloak in Tomcat server In-Reply-To: References: Message-ID: Hi, You do not need to deploy the server to Tomcat to be able to use Keycloak in your apps deployed to Tomcat. We only support deploying Keycloak server to latest version of WildFly and JBoss EAP. Further we highly recommend using the standalone download of Keycloak and running it separately to your applications. On 26 November 2015 at 07:24, mohammed althaf wrote: > Hi Team, > > I was able to use keycloak for sso seamlessly in jboss /wildfly server. > > Since my web application is running in tomcat ,i am trying to deploy the > keycloak server as well in Tomcat, > > I followed > https://reachmnadeem.wordpress.com/2015/01/14/deploying-keycloak-in-tomcat/ > > and i was able to spin up the keycloak in tomcat server.But it was using > the 1.0.2 version. > > So i am trying to deploy the keycloak server 1.6.1.I faced the below > issues. > > 1.Failed to provide mem for realmcache(please check failed to find > provider log) > > So i removed the cache from keycloak-server.json > Not sure it is fine?? > > 2 .Then i was able to procced but it again failed with h2.db > exception(please check lockout.log) > > It will be great if you guys can help me resolve this issue. > > Thanks, > Althaf > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151126/35e4e923/attachment.html From althaf1088 at gmail.com Thu Nov 26 02:54:49 2015 From: althaf1088 at gmail.com (mohammed althaf) Date: Thu, 26 Nov 2015 15:54:49 +0800 Subject: [keycloak-dev] Keycloak in Tomcat server In-Reply-To: References: Message-ID: Hi Stian, Thanks for your quick reply.Yeah i have tried the Tomcat adapters for keycloak Integration. I just want to explore the option to deploy it in tomcat itself ,since in enterprise if they already use tomcat for applications they wont recommend to use another server for SSO alone. >From your knowledge has anyone achieved that ?? Thanks, Althaf On 26 November 2015 at 15:40, Stian Thorgersen wrote: > Hi, > > You do not need to deploy the server to Tomcat to be able to use Keycloak > in your apps deployed to Tomcat. > > We only support deploying Keycloak server to latest version of WildFly and > JBoss EAP. Further we highly recommend using the standalone download of > Keycloak and running it separately to your applications. > > On 26 November 2015 at 07:24, mohammed althaf > wrote: > >> Hi Team, >> >> I was able to use keycloak for sso seamlessly in jboss /wildfly server. >> >> Since my web application is running in tomcat ,i am trying to deploy the >> keycloak server as well in Tomcat, >> >> I followed >> >> https://reachmnadeem.wordpress.com/2015/01/14/deploying-keycloak-in-tomcat/ >> >> and i was able to spin up the keycloak in tomcat server.But it was using >> the 1.0.2 version. >> >> So i am trying to deploy the keycloak server 1.6.1.I faced the below >> issues. >> >> 1.Failed to provide mem for realmcache(please check failed to find >> provider log) >> >> So i removed the cache from keycloak-server.json >> Not sure it is fine?? >> >> 2 .Then i was able to procced but it again failed with h2.db >> exception(please check lockout.log) >> >> It will be great if you guys can help me resolve this issue. >> >> Thanks, >> Althaf >> >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151126/e430da5c/attachment.html From sthorger at redhat.com Thu Nov 26 03:05:23 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 26 Nov 2015 09:05:23 +0100 Subject: [keycloak-dev] Keycloak in Tomcat server In-Reply-To: References: Message-ID: In enterprise they should recommend using a standalone SSO server. It's more secure and scalable. You wouldn't embed your database into Tomcat would you? It's also more reliable and less error prone as you are using the exact bits that we test. It has been achieved in the past, but initially we relied less on features from WildFly. Keycloak server is no longer a WAR that's deployed onto the WildFly app server, instead we rely on the core bits of WildFly to create our own dedicated server. In theory it would be possible to get it to work on top of Tomcat with some considerable amount of effort. However, as we add new features in the future that rely on features in WildFly you would have new issues. Further we would not be able to help you if you have any issues. On 26 November 2015 at 08:54, mohammed althaf wrote: > Hi Stian, > > Thanks for your quick reply.Yeah i have tried the Tomcat adapters for > keycloak Integration. > > I just want to explore the option to deploy it in tomcat itself ,since in > enterprise if they already use tomcat for applications they wont recommend > to use another server for SSO alone. > > From your knowledge has anyone achieved that ?? > > Thanks, > Althaf > > On 26 November 2015 at 15:40, Stian Thorgersen > wrote: > >> Hi, >> >> You do not need to deploy the server to Tomcat to be able to use Keycloak >> in your apps deployed to Tomcat. >> >> We only support deploying Keycloak server to latest version of WildFly >> and JBoss EAP. Further we highly recommend using the standalone download of >> Keycloak and running it separately to your applications. >> >> On 26 November 2015 at 07:24, mohammed althaf >> wrote: >> >>> Hi Team, >>> >>> I was able to use keycloak for sso seamlessly in jboss /wildfly server. >>> >>> Since my web application is running in tomcat ,i am trying to deploy the >>> keycloak server as well in Tomcat, >>> >>> I followed >>> >>> https://reachmnadeem.wordpress.com/2015/01/14/deploying-keycloak-in-tomcat/ >>> >>> and i was able to spin up the keycloak in tomcat server.But it was using >>> the 1.0.2 version. >>> >>> So i am trying to deploy the keycloak server 1.6.1.I faced the below >>> issues. >>> >>> 1.Failed to provide mem for realmcache(please check failed to find >>> provider log) >>> >>> So i removed the cache from keycloak-server.json >>> Not sure it is fine?? >>> >>> 2 .Then i was able to procced but it again failed with h2.db >>> exception(please check lockout.log) >>> >>> It will be great if you guys can help me resolve this issue. >>> >>> Thanks, >>> Althaf >>> >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151126/a24d8660/attachment.html From lkrzyzan at redhat.com Thu Nov 26 08:57:41 2015 From: lkrzyzan at redhat.com (Libor Krzyzanek) Date: Thu, 26 Nov 2015 14:57:41 +0100 Subject: [keycloak-dev] openid-connect/login-status-iframe.html caching in browser and on CDN Message-ID: <851C3E08-C8CF-4A5C-B3B5-8AF54CCA2FF5@redhat.com> Hi, I?m wondering if it would be possible to cache all unique ?.../openid-connect/login-status-iframe.html" in browser and then also on CDN e.g. Akamai. I mean whole URL with all parameters ?clinet_id? and ?origin?. This would greatly improve performance of website using JS client because Akamai servers content much faster. I checked our instance and http header has Cache-Control:max-age=2592000 which is 30 days Does it mean that we can configure Akamai to cache it? When the content of iframe.html could change and thus it?s needed to be invalidated on browser cache and on CDN cache? Thanks, Libor Krzy?anek jboss.org Development Team From bburke at redhat.com Thu Nov 26 09:19:36 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 26 Nov 2015 09:19:36 -0500 Subject: [keycloak-dev] openid-connect/login-status-iframe.html caching in browser and on CDN In-Reply-To: <851C3E08-C8CF-4A5C-B3B5-8AF54CCA2FF5@redhat.com> References: <851C3E08-C8CF-4A5C-B3B5-8AF54CCA2FF5@redhat.com> Message-ID: <565714F8.6060203@redhat.com> You can't. This request is done to determine if the user is logged in or not, IIRC. On 11/26/2015 8:57 AM, Libor Krzyzanek wrote: > Hi, > I?m wondering if it would be possible to cache all unique ?.../openid-connect/login-status-iframe.html" in browser and then also on CDN e.g. Akamai. > I mean whole URL with all parameters ?clinet_id? and ?origin?. > > This would greatly improve performance of website using JS client because Akamai servers content much faster. > > I checked our instance and http header has Cache-Control:max-age=2592000 which is 30 days > > Does it mean that we can configure Akamai to cache it? > When the content of iframe.html could change and thus it?s needed to be invalidated on browser cache and on CDN cache? > > Thanks, > > Libor Krzy?anek > jboss.org Development Team > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Thu Nov 26 09:26:44 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Thu, 26 Nov 2015 15:26:44 +0100 Subject: [keycloak-dev] openid-connect/login-status-iframe.html caching in browser and on CDN In-Reply-To: <565714F8.6060203@redhat.com> References: <851C3E08-C8CF-4A5C-B3B5-8AF54CCA2FF5@redhat.com> <565714F8.6060203@redhat.com> Message-ID: It can be cached. The contents is only the html for the iframe and doesn't check if user is logged in. The check if a user is logged in is done by javascript, which reads the value of the session state cookie. The only issue with caching is if the client is deleted or the origin is removed the session iframe is still valid until it's expired from the cache, but I don't think that's a problem. On 26 November 2015 at 15:19, Bill Burke wrote: > You can't. This request is done to determine if the user is logged in > or not, IIRC. > > On 11/26/2015 8:57 AM, Libor Krzyzanek wrote: > > Hi, > > I?m wondering if it would be possible to cache all unique > ?.../openid-connect/login-status-iframe.html" in browser and then also on > CDN e.g. Akamai. > > I mean whole URL with all parameters ?clinet_id? and ?origin?. > > > > This would greatly improve performance of website using JS client > because Akamai servers content much faster. > > > > I checked our instance and http header has Cache-Control:max-age=2592000 > which is 30 days > > > > Does it mean that we can configure Akamai to cache it? > > When the content of iframe.html could change and thus it?s needed to be > invalidated on browser cache and on CDN cache? > > > > Thanks, > > > > Libor Krzy?anek > > jboss.org Development Team > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151126/d03e3de9/attachment-0001.html From lkrzyzan at redhat.com Thu Nov 26 09:40:54 2015 From: lkrzyzan at redhat.com (Libor Krzyzanek) Date: Thu, 26 Nov 2015 15:40:54 +0100 Subject: [keycloak-dev] openid-connect/login-status-iframe.html caching in browser and on CDN In-Reply-To: References: <851C3E08-C8CF-4A5C-B3B5-8AF54CCA2FF5@redhat.com> <565714F8.6060203@redhat.com> Message-ID: That?s exactly my understanding. Thanks for clearing the impact in case that client is deleted/origin removed. Thanks, Libor Krzy?anek jboss.org Development Team > On Nov 26, 2015, at 3:26 PM, Stian Thorgersen wrote: > > It can be cached. The contents is only the html for the iframe and doesn't check if user is logged in. The check if a user is logged in is done by javascript, which reads the value of the session state cookie. > > The only issue with caching is if the client is deleted or the origin is removed the session iframe is still valid until it's expired from the cache, but I don't think that's a problem. > > On 26 November 2015 at 15:19, Bill Burke > wrote: > You can't. This request is done to determine if the user is logged in > or not, IIRC. > > On 11/26/2015 8:57 AM, Libor Krzyzanek wrote: > > Hi, > > I?m wondering if it would be possible to cache all unique ?.../openid-connect/login-status-iframe.html" in browser and then also on CDN e.g. Akamai. > > I mean whole URL with all parameters ?clinet_id? and ?origin?. > > > > This would greatly improve performance of website using JS client because Akamai servers content much faster. > > > > I checked our instance and http header has Cache-Control:max-age=2592000 which is 30 days > > > > Does it mean that we can configure Akamai to cache it? > > When the content of iframe.html could change and thus it?s needed to be invalidated on browser cache and on CDN cache? > > > > Thanks, > > > > Libor Krzy?anek > > jboss.org Development Team > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151126/1573276f/attachment.html From bruno at abstractj.org Thu Nov 26 15:24:25 2015 From: bruno at abstractj.org (Bruno Oliveira) Date: Thu, 26 Nov 2015 18:24:25 -0200 Subject: [keycloak-dev] User registration: OTP flow Message-ID: Ahoy, I was looking at this Jira https://issues.jboss.org/browse/KEYCLOAK-1998 and trying to reproduce the issue reported by Stian. What I did was: 1. Get the latest changes from master 2. Run mvn clean install -DskipTests=true && mvn -f testsuite/integration/pom.xml exec:java -Pkeycloak-server 3. Go to "Realm Settings" and enable "User Registration" 4. Go to "Authentication" > "Required Actions" 5. On Configure Totp mark the checkbox "Default Action" 6. Now logout and try to register 7. After the registration I get the TOTP screen Stian was following a different workflow 1. Get the latest changes from master 2. Run mvn clean install -DskipTests=true && mvn -f testsuite/integration/pom.xml exec:java -Pkeycloak-server 3. Go to "Realm Settings" and enable "User Registration" 4. Flows > Browser 5. OTP form marked as required 6. After the registration I won't get the TOTP screen After I managed to reproduce the real issue, I got confused about what would be the expected behavior in the situation where I have "OTP form" as required and "Required actions > Configure Totp > Default action" unchecked. To me it seems like OTP form is unnecessary, but I can be 110% wrong. Thoughts? From replymenot at gmail.com Thu Nov 26 17:00:45 2015 From: replymenot at gmail.com (replymenot) Date: Thu, 26 Nov 2015 23:00:45 +0100 Subject: [keycloak-dev] angular-demo-app Message-ID: <864C31C5-2294-4AE2-AAF2-F1FD955C54CB@gmail.com> I?m trying to get the angular demo app working and I got it to work but the logout url tries to go here: /realms/demo/tokens/logout?redirect_uri= I?ve loaded the demo realm but it has no ?tokens/logout? url What am I doing wrong? Greetings, Ivo. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151126/8d3adcd0/attachment.html From bburke at redhat.com Thu Nov 26 20:21:35 2015 From: bburke at redhat.com (Bill Burke) Date: Thu, 26 Nov 2015 20:21:35 -0500 Subject: [keycloak-dev] openid-connect/login-status-iframe.html caching in browser and on CDN In-Reply-To: References: <851C3E08-C8CF-4A5C-B3B5-8AF54CCA2FF5@redhat.com> <565714F8.6060203@redhat.com> Message-ID: <5657B01F.6080008@redhat.com> On 11/26/2015 9:26 AM, Stian Thorgersen wrote: > It can be cached. The contents is only the html for the iframe and > doesn't check if user is logged in. The check if a user is logged in is > done by javascript, which reads the value of the session state cookie. > Yeah, you're right sorry...funny cuz I implemented this stuff too... -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Fri Nov 27 02:40:19 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 08:40:19 +0100 Subject: [keycloak-dev] User registration: OTP flow In-Reply-To: References: Message-ID: It's certainly a usability issue then. I thought the opposite that "OTP Form" was required, but not adding the default action to have it work with registration as well. On 26 November 2015 at 21:24, Bruno Oliveira wrote: > Ahoy, I was looking at this Jira > https://issues.jboss.org/browse/KEYCLOAK-1998 and trying to reproduce > the issue reported by Stian. > > What I did was: > > 1. Get the latest changes from master > 2. Run mvn clean install -DskipTests=true && mvn -f > testsuite/integration/pom.xml exec:java -Pkeycloak-server > 3. Go to "Realm Settings" and enable "User Registration" > 4. Go to "Authentication" > "Required Actions" > 5. On Configure Totp mark the checkbox "Default Action" > 6. Now logout and try to register > 7. After the registration I get the TOTP screen > > Stian was following a different workflow > > 1. Get the latest changes from master > 2. Run mvn clean install -DskipTests=true && mvn -f > testsuite/integration/pom.xml exec:java -Pkeycloak-server > 3. Go to "Realm Settings" and enable "User Registration" > 4. Flows > Browser > 5. OTP form marked as required > 6. After the registration I won't get the TOTP screen > > After I managed to reproduce the real issue, I got confused about what > would be the expected behavior in the situation where I have "OTP > form" as required and "Required actions > Configure Totp > Default > action" unchecked. > > To me it seems like OTP form is unnecessary, but I can be 110% wrong. > > Thoughts? > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/660a20b3/attachment.html From sthorger at redhat.com Fri Nov 27 02:41:26 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 08:41:26 +0100 Subject: [keycloak-dev] openid-connect/login-status-iframe.html caching in browser and on CDN In-Reply-To: <5657B01F.6080008@redhat.com> References: <851C3E08-C8CF-4A5C-B3B5-8AF54CCA2FF5@redhat.com> <565714F8.6060203@redhat.com> <5657B01F.6080008@redhat.com> Message-ID: That's like 2 years ago though! On 27 November 2015 at 02:21, Bill Burke wrote: > > > On 11/26/2015 9:26 AM, Stian Thorgersen wrote: > >> It can be cached. The contents is only the html for the iframe and >> doesn't check if user is logged in. The check if a user is logged in is >> done by javascript, which reads the value of the session state cookie. >> >> > Yeah, you're right sorry...funny cuz I implemented this stuff too... > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/184d7346/attachment-0001.html From mposolda at redhat.com Fri Nov 27 03:28:03 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 27 Nov 2015 09:28:03 +0100 Subject: [keycloak-dev] Implicit and hybrid flow Message-ID: <56581413.4010709@redhat.com> I've sent PR https://github.com/keycloak/keycloak/pull/1885 with initial support for implicit flow and hybrid flow. Some summary: - Added switches on client in admin console to enable/disable standard flow, implicit flow, direct grant flow and service accounts. Removed "direct grants only" switch (Disable both standard and implicit defacto means enabling of previous "direct grants only") - Added more proper support for "response_type" parameter. This is about _what_ is sent in response from AuthorizationEndpoint to client application. According to specs, possible values are "code", "id_token", "token" and some combination of them. See [1] . Until now, we supported just value "code" (standard AuthorizationCode flow). According to specs, implicit flow is about "token" and "id_token". Hybrid flow is about code + some of the tokens. In addition to specs, I've added also support for "refresh_token" . So if you use "response_type=token%20id_token%20refresh_token", the AuthorizationEndpoint will send all 3 tokens. The OIDC specs doesn't support sending refreshToken in implicit flow. However in Keycloak case, we are using access tokens with very short lifespan by default (1 minute). Having just accessToken and idToken would make the implicit flow quite unusable, as application will need to re-login each 1 minute. For example Google doesn't support sending refreshToken in implicit flow, however it makes much more sense for them as their access token is valid for 60 minutes. This is even longer than our refresh token (30 minutes by default). WDYT about support for refresh token? - Added support for "response_mode" parameter. This specifies _how_ are OIDC parameters (code, state, ... ) sent from AuthorizationEndpoint to client. More details in specs [2]. Valid values are: -- query - Params sent in query string ( http://localhost?code=foo&state=bar ). We always supported this -- fragment - Params sent in fragment ( http://localhost#code=foo&state=bar ). This is needed for implicit and hybrid flow support. Specs doesn't allow to sent tokens in query string. -- form_post - Params are sent in body of POST method. There is separate specs for this [3] . I've added just server-side support for this. It may be interesting to add support on our server adapters, as IMO it's a bit safer alternative when code+state are sent in POST body instead of in GET method query string. But I guess this is not big priority? - keycloak.js changes - I've added 2 new init config options. Option "response_mode" with possible values "query" and "fragment" (see above what they mean). Second is option "flow" with possible values: a) "standard" - will use response_type=code . This is what we always supported b) "implicit" - will use response_type=id_token%20token%20refresh_token (So sending all 3 tokens) c) "hybrid" - will use response_type=code%20id_token%20token%20refresh_token (Sending code and all 3 tokens). Not sure if we really need to support "hybrid" option. Specs provides hybrid flow for semi-confidential application, which can decide if they need refreshToken (in that case app must send request to exchange code) or if they don't need refresh token and access+id tokens are sufficient. But in Keycloak, since we support sending refresh_token directly in implicit flow (see above), this flow is not very useful for us. Default values for options are response_type=code and response_mode=fragment. So we still use "code" and standard flow by default, however we sent code+state in fragment now. This is better and safer than query string. In qery string, the code+state were always sent in request to the client application. This is not needed, keycloak.js needs them available just on browser side. With fragment, code and state are not sent to client application, so it's one less possibility how they can be compromised. - Added support for "nonce" check in keycloak.js. It's useful for some kind of attacks and specs wants it for implicit flow. Possible pending work: - Fix integration-arquillian as PR is failing now - Possibly add server-side support for "at_hash" to IDToken ? Not sure if at_hash support has any real advantages for us, however specs requires that and even Google supports it. - Add docs and tests. Not sure about adding separate example? Until now, I've tested with existing js-console example and added support here for easily set init parameters to keycloak.init, so people can use different flow or responseMode here. WDYT? Marek [1] http://openid.net/specs/oauth-v2-multiple-response-types-1_0.html#Combinations [2] http://openid.net/specs/oauth-v2-multiple-response-types-1_0.html#ResponseModes [3] http://openid.net/specs/oauth-v2-form-post-response-mode-1_0.html From sthorger at redhat.com Fri Nov 27 03:29:11 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 09:29:11 +0100 Subject: [keycloak-dev] Reset Actions Message-ID: The new reset actions doesn't require the user to authenticate prior to performing them. Is it not a bit dangerous that the user can change the email address without authentication? For reset password we obviously need to be able to do it without requiring authentication, but shouldn't "bypassing" authentication be limited as much as possible? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/29abda5f/attachment.html From sthorger at redhat.com Fri Nov 27 03:44:31 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 09:44:31 +0100 Subject: [keycloak-dev] Implicit and hybrid flow In-Reply-To: <56581413.4010709@redhat.com> References: <56581413.4010709@redhat.com> Message-ID: On 27 November 2015 at 09:28, Marek Posolda wrote: > I've sent PR https://github.com/keycloak/keycloak/pull/1885 with initial > support for implicit flow and hybrid flow. > > Some summary: > - Added switches on client in admin console to enable/disable standard > flow, implicit flow, direct grant flow and service accounts. Removed > "direct grants only" switch (Disable both standard and implicit defacto > means enabling of previous "direct grants only") > Is direct grant and implicit disabled by default? > > > - Added more proper support for "response_type" parameter. This is about > _what_ is sent in response from AuthorizationEndpoint to client > application. According to specs, possible values are "code", "id_token", > "token" and some combination of them. See [1] . Until now, we supported > just value "code" (standard AuthorizationCode flow). According to specs, > implicit flow is about "token" and "id_token". Hybrid flow is about code > + some of the tokens. > What do we do about backwards compatibility? It's seems we're breaking the spec currently by including both id_token and refresh_token even though response_type is just code. > > In addition to specs, I've added also support for "refresh_token" . So > if you use "response_type=token%20id_token%20refresh_token", the > AuthorizationEndpoint will send all 3 tokens. The OIDC specs doesn't > support sending refreshToken in implicit flow. However in Keycloak case, > we are using access tokens with very short lifespan by default (1 > minute). Having just accessToken and idToken would make the implicit > flow quite unusable, as application will need to re-login each 1 minute. > For example Google doesn't support sending refreshToken in implicit > flow, however it makes much more sense for them as their access token is > valid for 60 minutes. This is even longer than our refresh token (30 > minutes by default). WDYT about support for refresh token? > We shouldn't send refresh token in fragment aka support it in implicit. Maybe we should have different timeouts instead of sending refresh token? > > > - Added support for "response_mode" parameter. This specifies _how_ are > OIDC parameters (code, state, ... ) sent from AuthorizationEndpoint to > client. More details in specs [2]. Valid values are: > -- query - Params sent in query string ( > http://localhost?code=foo&state=bar ). We always supported this > -- fragment - Params sent in fragment ( > http://localhost#code=foo&state=bar ). This is needed for implicit and > hybrid flow support. Specs doesn't allow to sent tokens in query string. > -- form_post - Params are sent in body of POST method. There is separate > specs for this [3] . I've added just server-side support for this. It > may be interesting to add support on our server adapters, as IMO it's a > bit safer alternative when code+state are sent in POST body instead of > in GET method query string. But I guess this is not big priority? > Maybe just JIRA it for 1.8 > > > - keycloak.js changes - I've added 2 new init config options. Option > "response_mode" with possible values "query" and "fragment" (see above > what they mean). Second is option "flow" with possible values: > a) "standard" - will use response_type=code . This is what we always > supported > b) "implicit" - will use response_type=id_token%20token%20refresh_token > (So sending all 3 tokens) > c) "hybrid" - will use > response_type=code%20id_token%20token%20refresh_token (Sending code and > all 3 tokens). Not sure if we really need to support "hybrid" option. > Specs provides hybrid flow for semi-confidential application, which can > decide if they need refreshToken (in that case app must send request to > exchange code) or if they don't need refresh token and access+id tokens > are sufficient. But in Keycloak, since we support sending refresh_token > directly in implicit flow (see above), this flow is not very useful for us. > Actually sounds like hybrid is better if we want to have refresh token in keycloak.js. With hybrid you don't need the first request to get access token, but at the same time refresh token isn't sent in the fragment. > > Default values for options are response_type=code and > response_mode=fragment. So we still use "code" and standard flow by > default, however we sent code+state in fragment now. This is better and > safer than query string. In qery string, the code+state were always sent > in request to the client application. This is not needed, keycloak.js > needs them available just on browser side. With fragment, code and state > are not sent to client application, so it's one less possibility how > they can be compromised. > > - Added support for "nonce" check in keycloak.js. It's useful for some > kind of attacks and specs wants it for implicit flow. > > Possible pending work: > - Fix integration-arquillian as PR is failing now > > - Possibly add server-side support for "at_hash" to IDToken ? Not sure > if at_hash support has any real advantages for us, however specs > requires that and even Google supports it. > > - Add docs and tests. Not sure about adding separate example? Until now, > I've tested with existing js-console example and added support here for > easily set init parameters to keycloak.init, so people can use different > flow or responseMode here. > Docs is enough. I don't think there's any need for examples. > > WDYT? > > Marek > > [1] > > http://openid.net/specs/oauth-v2-multiple-response-types-1_0.html#Combinations > > [2] > > http://openid.net/specs/oauth-v2-multiple-response-types-1_0.html#ResponseModes > > [3] http://openid.net/specs/oauth-v2-form-post-response-mode-1_0.html > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/6c390964/attachment.html From velias at redhat.com Fri Nov 27 04:23:25 2015 From: velias at redhat.com (Vlastimil Elias) Date: Fri, 27 Nov 2015 10:23:25 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app Message-ID: <5658210D.1070301@redhat.com> Hi, I have two proposals for cleanup of 'Change password' screen in Account app based on my experience with it: 1. remove Cancel button - it has no any meaning on this screen/form, it only reshowns form with empty fields. And also there is a bug, "Password" field is hidden when it is used, which makes whole form unusable. 2. remove validation of current password (remove "Password" field). Two reasons for this: - security impact of this check is small. If attacker is able to compromise Account app then he can always change email and then use "Forgot password" feature to change password - user created over Identity Provider do not know old password (because it is not set) so he is not able to set password using this screen After we implement support for reauthentication (KEYCLOAK-2076) then we should set some reasonable reauth timeout for Account app instead, this will make it more secure at all. If you agree then I can create JIRA issue for this and provide PR. Vlastimil -- Vlastimil Elias Principal Software Engineer Developer Portal Engineering Team From mposolda at redhat.com Fri Nov 27 04:53:23 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 27 Nov 2015 10:53:23 +0100 Subject: [keycloak-dev] Implicit and hybrid flow In-Reply-To: References: <56581413.4010709@redhat.com> Message-ID: <56582813.3010501@redhat.com> On 27/11/15 09:44, Stian Thorgersen wrote: > > > On 27 November 2015 at 09:28, Marek Posolda > wrote: > > I've sent PR https://github.com/keycloak/keycloak/pull/1885 with > initial > support for implicit flow and hybrid flow. > > Some summary: > - Added switches on client in admin console to enable/disable standard > flow, implicit flow, direct grant flow and service accounts. Removed > "direct grants only" switch (Disable both standard and implicit > defacto > means enabling of previous "direct grants only") > > > Is direct grant and implicit disabled by default? Implicit is disabled, but direct grant is enabled by default. This is just for backwards compatibility, as in 1.6, we have direct grant defacto enabled for all clients. If we want to have it disabled by default, we should add big note to migration docs. Or we can have it enabled for all clients migrated from previous version, but keep the switch "off" in admin console for new clients? At least, we have people, who wants to login into admin REST API by default (without need to go to admin-console UI first and enable direct grant for some client), so I guess this possibility should be still kept. > > > > - Added more proper support for "response_type" parameter. This is > about > _what_ is sent in response from AuthorizationEndpoint to client > application. According to specs, possible values are "code", > "id_token", > "token" and some combination of them. See [1] . Until now, we > supported > just value "code" (standard AuthorizationCode flow). According to > specs, > implicit flow is about "token" and "id_token". Hybrid flow is > about code > + some of the tokens. > > > What do we do about backwards compatibility? It's seems we're breaking > the spec currently by including both id_token and refresh_token even > though response_type is just code. No, we include just what is requested in response_type parameter. If it's just "code", we will send just "code" (as it always was), so the behaviour for response_type=code is unchanged and completely backwards compatible. If it's for example "code id_token" we send code + id_token . Nothing more. Using of any of tokens parameters also requires implicit flow enabled for the client. Hybrid flow is allowed just if both "standard" and "implicit" are enabled for the client. > > > In addition to specs, I've added also support for "refresh_token" . So > if you use "response_type=token%20id_token%20refresh_token", the > AuthorizationEndpoint will send all 3 tokens. The OIDC specs doesn't > support sending refreshToken in implicit flow. However in Keycloak > case, > we are using access tokens with very short lifespan by default (1 > minute). Having just accessToken and idToken would make the implicit > flow quite unusable, as application will need to re-login each 1 > minute. > For example Google doesn't support sending refreshToken in implicit > flow, however it makes much more sense for them as their access > token is > valid for 60 minutes. This is even longer than our refresh token (30 > minutes by default). WDYT about support for refresh token? > > > We shouldn't send refresh token in fragment aka support it in > implicit. Maybe we should have different timeouts instead of sending > refresh token? Ok, working on it as we discussed privately. > > > > - Added support for "response_mode" parameter. This specifies > _how_ are > OIDC parameters (code, state, ... ) sent from AuthorizationEndpoint to > client. More details in specs [2]. Valid values are: > -- query - Params sent in query string ( > http://localhost?code=foo&state=bar ). We always supported this > -- fragment - Params sent in fragment ( > http://localhost#code=foo&state=bar ). This is needed for implicit and > hybrid flow support. Specs doesn't allow to sent tokens in query > string. > -- form_post - Params are sent in body of POST method. There is > separate > specs for this [3] . I've added just server-side support for this. It > may be interesting to add support on our server adapters, as IMO > it's a > bit safer alternative when code+state are sent in POST body instead of > in GET method query string. But I guess this is not big priority? > > > Maybe just JIRA it for 1.8 https://issues.jboss.org/browse/KEYCLOAK-2153 > > > > - keycloak.js changes - I've added 2 new init config options. Option > "response_mode" with possible values "query" and "fragment" (see above > what they mean). Second is option "flow" with possible values: > a) "standard" - will use response_type=code . This is what we always > supported > b) "implicit" - will use > response_type=id_token%20token%20refresh_token > (So sending all 3 tokens) > c) "hybrid" - will use > response_type=code%20id_token%20token%20refresh_token (Sending > code and > all 3 tokens). Not sure if we really need to support "hybrid" option. > Specs provides hybrid flow for semi-confidential application, > which can > decide if they need refreshToken (in that case app must send > request to > exchange code) or if they don't need refresh token and access+id > tokens > are sufficient. But in Keycloak, since we support sending > refresh_token > directly in implicit flow (see above), this flow is not very > useful for us. > > > Actually sounds like hybrid is better if we want to have refresh token > in keycloak.js. With hybrid you don't need the first request to get > access token, but at the same time refresh token isn't sent in the > fragment. Yes, as long as we want implicit flow exactly according to specs (so no refresh_token in fragment) then hybrid is useful as the code needs to be used to exchange for refresh_token . > > > Default values for options are response_type=code and > response_mode=fragment. So we still use "code" and standard flow by > default, however we sent code+state in fragment now. This is > better and > safer than query string. In qery string, the code+state were > always sent > in request to the client application. This is not needed, keycloak.js > needs them available just on browser side. With fragment, code and > state > are not sent to client application, so it's one less possibility how > they can be compromised. > > - Added support for "nonce" check in keycloak.js. It's useful for some > kind of attacks and specs wants it for implicit flow. > > Possible pending work: > - Fix integration-arquillian as PR is failing now > > - Possibly add server-side support for "at_hash" to IDToken ? Not sure > if at_hash support has any real advantages for us, however specs > requires that and even Google supports it. > > - Add docs and tests. Not sure about adding separate example? > Until now, > I've tested with existing js-console example and added support > here for > easily set init parameters to keycloak.init, so people can use > different > flow or responseMode here. > > > Docs is enough. I don't think there's any need for examples. > > > WDYT? > > Marek > > [1] > http://openid.net/specs/oauth-v2-multiple-response-types-1_0.html#Combinations > > [2] > http://openid.net/specs/oauth-v2-multiple-response-types-1_0.html#ResponseModes > > [3] http://openid.net/specs/oauth-v2-form-post-response-mode-1_0.html > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/d898f502/attachment-0001.html From velias at redhat.com Fri Nov 27 05:05:34 2015 From: velias at redhat.com (Vlastimil Elias) Date: Fri, 27 Nov 2015 11:05:34 +0100 Subject: [keycloak-dev] [keycloak-user] How to implement long user sso sessions with reauthentication for important actions? In-Reply-To: References: <5644910D.9020100@redhat.com> <564498DE.6010805@redhat.com> Message-ID: <56582AEE.4020508@redhat.com> Hi, moving this discussion to the devel forum as it is about the feture development now. Toplevel issue I created for this feature is https://issues.jboss.org/browse/KEYCLOAK-2076 I added some notes and thoughts from my investigation as a comment to the KEYCLOAK-2076, there are some open questions how to implement it. Originally I though I should be able to implement reauth support and provide PR. But I must say I'm not sure now if I'm able to implement it, looks like it is a bit more complicated than I originally expected, so probably some Keycloak core developer should do it. But if you think you will not have resources to do it in 1.8 then I can try it (with your support), as I believe it is a very important feature, and we really want use it. Cheers Vlastimil On 12.11.2015 14:50, Stian Thorgersen wrote: > > > On 12 November 2015 at 14:49, Vlastimil Elias > wrote: > > Thanks for quick reply Stian. > > I'm going to create JIRAs for all these things. I can volunter to > implement some parts of this. > > For the last one, it should be probably cool to have > "reauthenticate timeout" setting available in client section for > every client (not only internal admin console and account > management). It should allow simple implementation of "long user > sso session" scheme even in environments where some clients can't > be updated to set max_age on protocol level. > > > Yep, that makes sense > > > > Vl. > > > On 12.11.2015 14:39, Stian Thorgersen wrote: >> >> >> On 12 November 2015 at 14:15, Vlastimil Elias > > wrote: >> >> Hi, >> >> I'd like to use long session authentication mechanism known >> from many >> sites like google. facebook, linked in etc. >> It is about really long user SSO sessions (eg. weeks or even >> months) >> with reauthentication for important actions when last >> authentication >> timestamp is older than some limit. >> >> Is this somehow possible with current Keycloak server and >> Keycloak adapters? >> >> I see few subquestions in this problem for our use: >> >> ***** >> open-id connect protocol defines few auth request parameters >> to support >> this use case, mainly max_age or prompt=login. Are they correctly >> implemented in Keycloak server? >> >> >> We don't have support for max_age and we only support prompt=none >> so these would have to be added >> >> >> >> >> ***** >> Wildfly/EAP adapter - is it possible and is there some >> example how to >> use "reauth if auth is older than 30min" action in Java app >> secured by >> this adapter? Or is info about last auth timestamp somehow >> available in >> the app? >> >> >> We don't set auth_time claim ATM so answer is no >> >> >> >> >> ***** >> Keycloak user account application itself - it is part of the >> Keycloak >> server, but it contains sensitive actions which typically require >> reathentication in this long session scheme (password change, >> email >> change, ...). Is it somehow possible to configure Keycloak to >> force >> timeout reauth for this app? >> >> >> Not at the moment - but if we add what you want it would also >> make sense to add that. Would need to be configurable through the >> admin console. Would also be nice to have the same for the admin >> console itself. >> >> >> >> Thanks in advance >> >> Vl. >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> >> >> _______________________________________________ >> keycloak-user mailing list >> keycloak-user at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-user >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -- Vlastimil Elias Principal Software Engineer Developer Portal Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/d1656c44/attachment.html From thomas.raehalme at aitiofinland.com Fri Nov 27 05:05:43 2015 From: thomas.raehalme at aitiofinland.com (Thomas Raehalme) Date: Fri, 27 Nov 2015 12:05:43 +0200 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: <5658210D.1070301@redhat.com> References: <5658210D.1070301@redhat.com> Message-ID: Hi! On Fri, Nov 27, 2015 at 11:23 AM, Vlastimil Elias wrote: > 2. remove validation of current password (remove "Password" field). Two > reasons for this: > - security impact of this check is small. If attacker is able to > compromise Account app then he can always change email and then use > "Forgot password" feature to change password > - user created over Identity Provider do not know old password > (because it is not set) so he is not able to set password using this screen > After we implement support for reauthentication (KEYCLOAK-2076) then we > should set some reasonable reauth timeout for Account app instead, this > will make it more secure at all. > Wouldn't it make more sense to add password validation when changing email? Best regards, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/380a0a5c/attachment.html From mposolda at redhat.com Fri Nov 27 05:14:19 2015 From: mposolda at redhat.com (Marek Posolda) Date: Fri, 27 Nov 2015 11:14:19 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: <5658210D.1070301@redhat.com> References: <5658210D.1070301@redhat.com> Message-ID: <56582CFB.3090107@redhat.com> On 27/11/15 10:23, Vlastimil Elias wrote: > Hi, > > I have two proposals for cleanup of 'Change password' screen in Account > app based on my experience with it: > > 1. remove Cancel button - it has no any meaning on this screen/form, it > only reshowns form with empty fields. And also there is a bug, > "Password" field is hidden when it is used, which makes whole form unusable. > > 2. remove validation of current password (remove "Password" field). Two > reasons for this: > - security impact of this check is small. If attacker is able to > compromise Account app then he can always change email and then use > "Forgot password" feature to change password > - user created over Identity Provider do not know old password > (because it is not set) so he is not able to set password using this screen > After we implement support for reauthentication (KEYCLOAK-2076) then we > should set some reasonable reauth timeout for Account app instead, this > will make it more secure at all. Hmm... AFAIK if user doesn't have password set, he is already not required to fill the existing password. That is the case when he is registered through some social/identity providers. See PasswordBean class. Btv. with new firstBrokerLogin changes, there is also flag "requirePasswordUpdateAfterRegistration" on CreateUserIfUnique authenticator. So now you can ask users registered through social to update the password immediately after they are registered through social provider :-) Marek > > If you agree then I can create JIRA issue for this and provide PR. > > Vlastimil > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/b94bdde7/attachment.html From velias at redhat.com Fri Nov 27 05:16:32 2015 From: velias at redhat.com (Vlastimil Elias) Date: Fri, 27 Nov 2015 11:16:32 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: References: <5658210D.1070301@redhat.com> Message-ID: <56582D80.4040608@redhat.com> Hi Thomas, On 27.11.2015 11:05, Thomas Raehalme wrote: > Hi! > > On Fri, Nov 27, 2015 at 11:23 AM, Vlastimil Elias > wrote: > > 2. remove validation of current password (remove "Password" > field). Two > reasons for this: > - security impact of this check is small. If attacker is able to > compromise Account app then he can always change email and then use > "Forgot password" feature to change password > - user created over Identity Provider do not know old password > (because it is not set) so he is not able to set password using > this screen > After we implement support for reauthentication (KEYCLOAK-2076) > then we > should set some reasonable reauth timeout for Account app instead, > this > will make it more secure at all. > > > Wouldn't it make more sense to add password validation when changing > email? Yes, this is why I write about use of general reauthentication mechanism as defined in KEYCLOAK-2076 for whole Account app. It will work even for other authentication types - some keycloak instances may be configured not to use passwords at all. Vl > > Best regards, > Thomas -- Vlastimil Elias Principal Software Engineer Developer Portal Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/32d655d8/attachment-0001.html From sthorger at redhat.com Fri Nov 27 05:45:10 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 11:45:10 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: <5658210D.1070301@redhat.com> References: <5658210D.1070301@redhat.com> Message-ID: On 27 November 2015 at 10:23, Vlastimil Elias wrote: > Hi, > > I have two proposals for cleanup of 'Change password' screen in Account > app based on my experience with it: > > 1. remove Cancel button - it has no any meaning on this screen/form, it > only reshowns form with empty fields. And also there is a bug, > "Password" field is hidden when it is used, which makes whole form > unusable. > +1 > > 2. remove validation of current password (remove "Password" field). Two > reasons for this: > - security impact of this check is small. If attacker is able to > compromise Account app then he can always change email and then use > "Forgot password" feature to change password > - user created over Identity Provider do not know old password > (because it is not set) so he is not able to set password using this screen > After we implement support for reauthentication (KEYCLOAK-2076) then we > should set some reasonable reauth timeout for Account app instead, this > will make it more secure at all. > -1 Reset password over email may not be enabled at all. We already allow setting password for IdPs login without requiring the existing password. +1 To suggestion from Thomas - we should ask for password when updating email at least when recover password over email is enabled. It seems to be common practice to ask for current password when updating the existing password. > > If you agree then I can create JIRA issue for this and provide PR. > > Vlastimil > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/bdc1473c/attachment.html From sthorger at redhat.com Fri Nov 27 05:46:07 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 11:46:07 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: References: <5658210D.1070301@redhat.com> Message-ID: +1 To setting re-auth after a configurable time for both admin console and account mngmt. I still think we should ask for password though. On 27 November 2015 at 11:45, Stian Thorgersen wrote: > > > On 27 November 2015 at 10:23, Vlastimil Elias wrote: > >> Hi, >> >> I have two proposals for cleanup of 'Change password' screen in Account >> app based on my experience with it: >> >> 1. remove Cancel button - it has no any meaning on this screen/form, it >> only reshowns form with empty fields. And also there is a bug, >> "Password" field is hidden when it is used, which makes whole form >> unusable. >> > > +1 > > >> >> 2. remove validation of current password (remove "Password" field). Two >> reasons for this: >> - security impact of this check is small. If attacker is able to >> compromise Account app then he can always change email and then use >> "Forgot password" feature to change password >> - user created over Identity Provider do not know old password >> (because it is not set) so he is not able to set password using this >> screen >> After we implement support for reauthentication (KEYCLOAK-2076) then we >> should set some reasonable reauth timeout for Account app instead, this >> will make it more secure at all. >> > > -1 Reset password over email may not be enabled at all. We already allow > setting password for IdPs login without requiring the existing password. > > +1 To suggestion from Thomas - we should ask for password when updating > email at least when recover password over email is enabled. > > It seems to be common practice to ask for current password when updating > the existing password. > > >> >> If you agree then I can create JIRA issue for this and provide PR. >> >> Vlastimil >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/02c6c0f1/attachment.html From sthorger at redhat.com Fri Nov 27 05:47:25 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 11:47:25 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: References: <5658210D.1070301@redhat.com> Message-ID: Please add suggestions to improve account management to https://issues.jboss.org/browse/KEYCLOAK-1250. We're hoping to completely rework the account management console as it's not very nice nor user friendly at the moment. On 27 November 2015 at 11:46, Stian Thorgersen wrote: > +1 To setting re-auth after a configurable time for both admin console and > account mngmt. I still think we should ask for password though. > > On 27 November 2015 at 11:45, Stian Thorgersen > wrote: > >> >> >> On 27 November 2015 at 10:23, Vlastimil Elias wrote: >> >>> Hi, >>> >>> I have two proposals for cleanup of 'Change password' screen in Account >>> app based on my experience with it: >>> >>> 1. remove Cancel button - it has no any meaning on this screen/form, it >>> only reshowns form with empty fields. And also there is a bug, >>> "Password" field is hidden when it is used, which makes whole form >>> unusable. >>> >> >> +1 >> >> >>> >>> 2. remove validation of current password (remove "Password" field). Two >>> reasons for this: >>> - security impact of this check is small. If attacker is able to >>> compromise Account app then he can always change email and then use >>> "Forgot password" feature to change password >>> - user created over Identity Provider do not know old password >>> (because it is not set) so he is not able to set password using this >>> screen >>> After we implement support for reauthentication (KEYCLOAK-2076) then we >>> should set some reasonable reauth timeout for Account app instead, this >>> will make it more secure at all. >>> >> >> -1 Reset password over email may not be enabled at all. We already allow >> setting password for IdPs login without requiring the existing password. >> >> +1 To suggestion from Thomas - we should ask for password when updating >> email at least when recover password over email is enabled. >> >> It seems to be common practice to ask for current password when updating >> the existing password. >> >> >>> >>> If you agree then I can create JIRA issue for this and provide PR. >>> >>> Vlastimil >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/d4961045/attachment.html From sthorger at redhat.com Fri Nov 27 05:52:43 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 11:52:43 +0100 Subject: [keycloak-dev] Implicit and hybrid flow In-Reply-To: <56582813.3010501@redhat.com> References: <56581413.4010709@redhat.com> <56582813.3010501@redhat.com> Message-ID: On 27 November 2015 at 10:53, Marek Posolda wrote: > On 27/11/15 09:44, Stian Thorgersen wrote: > > > > On 27 November 2015 at 09:28, Marek Posolda wrote: > >> I've sent PR https://github.com/keycloak/keycloak/pull/1885 with initial >> support for implicit flow and hybrid flow. >> >> Some summary: >> - Added switches on client in admin console to enable/disable standard >> flow, implicit flow, direct grant flow and service accounts. Removed >> "direct grants only" switch (Disable both standard and implicit defacto >> means enabling of previous "direct grants only") >> > > Is direct grant and implicit disabled by default? > > Implicit is disabled, but direct grant is enabled by default. This is just > for backwards compatibility, as in 1.6, we have direct grant defacto > enabled for all clients. If we want to have it disabled by default, we > should add big note to migration docs. Or we can have it enabled for all > clients migrated from previous version, but keep the switch "off" in admin > console for new clients? > On for old, off for new works for me. > > At least, we have people, who wants to login into admin REST API by > default (without need to go to admin-console UI first and enable direct > grant for some client), so I guess this possibility should be still kept. > In reality they should not be using the admin console client to do so. They should create a separate client for it I think. We need to sort out some sort of bootstrapping for it though. Or maybe we have a admin-cli client? > > > >> >> >> - Added more proper support for "response_type" parameter. This is about >> _what_ is sent in response from AuthorizationEndpoint to client >> application. According to specs, possible values are "code", "id_token", >> "token" and some combination of them. See [1] . Until now, we supported >> just value "code" (standard AuthorizationCode flow). According to specs, >> implicit flow is about "token" and "id_token". Hybrid flow is about code >> + some of the tokens. >> > > What do we do about backwards compatibility? It's seems we're breaking the > spec currently by including both id_token and refresh_token even though > response_type is just code. > > No, we include just what is requested in response_type parameter. If it's > just "code", we will send just "code" (as it always was), so the behaviour > for response_type=code is unchanged and completely backwards compatible. If > it's for example "code id_token" we send code + id_token . Nothing more. > Using of any of tokens parameters also requires implicit flow enabled for > the client. Hybrid flow is allowed just if both "standard" and "implicit" > are enabled for the client. > I was thinking the access token request had a parameter to request refresh token, but it doesn't. So you're right. > > > >> >> In addition to specs, I've added also support for "refresh_token" . So >> if you use "response_type=token%20id_token%20refresh_token", the >> AuthorizationEndpoint will send all 3 tokens. The OIDC specs doesn't >> support sending refreshToken in implicit flow. However in Keycloak case, >> we are using access tokens with very short lifespan by default (1 >> minute). Having just accessToken and idToken would make the implicit >> flow quite unusable, as application will need to re-login each 1 minute. >> For example Google doesn't support sending refreshToken in implicit >> flow, however it makes much more sense for them as their access token is >> valid for 60 minutes. This is even longer than our refresh token (30 >> minutes by default). WDYT about support for refresh token? >> > > We shouldn't send refresh token in fragment aka support it in implicit. > Maybe we should have different timeouts instead of sending refresh token? > > Ok, working on it as we discussed privately. > > > >> >> >> - Added support for "response_mode" parameter. This specifies _how_ are >> OIDC parameters (code, state, ... ) sent from AuthorizationEndpoint to >> client. More details in specs [2]. Valid values are: >> -- query - Params sent in query string ( >> http://localhost?code=foo&state=bar ). We always supported this >> -- fragment - Params sent in fragment ( >> http://localhost#code=foo&state=bar ). This is needed for implicit and >> hybrid flow support. Specs doesn't allow to sent tokens in query string. >> -- form_post - Params are sent in body of POST method. There is separate >> specs for this [3] . I've added just server-side support for this. It >> may be interesting to add support on our server adapters, as IMO it's a >> bit safer alternative when code+state are sent in POST body instead of >> in GET method query string. But I guess this is not big priority? >> > > Maybe just JIRA it for 1.8 > > https://issues.jboss.org/browse/KEYCLOAK-2153 > > > >> >> >> - keycloak.js changes - I've added 2 new init config options. Option >> "response_mode" with possible values "query" and "fragment" (see above >> what they mean). Second is option "flow" with possible values: >> a) "standard" - will use response_type=code . This is what we always >> supported >> b) "implicit" - will use response_type=id_token%20token%20refresh_token >> (So sending all 3 tokens) >> c) "hybrid" - will use >> response_type=code%20id_token%20token%20refresh_token (Sending code and >> all 3 tokens). Not sure if we really need to support "hybrid" option. >> Specs provides hybrid flow for semi-confidential application, which can >> decide if they need refreshToken (in that case app must send request to >> exchange code) or if they don't need refresh token and access+id tokens >> are sufficient. But in Keycloak, since we support sending refresh_token >> directly in implicit flow (see above), this flow is not very useful for >> us. >> > > Actually sounds like hybrid is better if we want to have refresh token in > keycloak.js. With hybrid you don't need the first request to get access > token, but at the same time refresh token isn't sent in the fragment. > > Yes, as long as we want implicit flow exactly according to specs (so no > refresh_token in fragment) then hybrid is useful as the code needs to be > used to exchange for refresh_token . > > > >> >> Default values for options are response_type=code and >> response_mode=fragment. So we still use "code" and standard flow by >> default, however we sent code+state in fragment now. This is better and >> safer than query string. In qery string, the code+state were always sent >> in request to the client application. This is not needed, keycloak.js >> needs them available just on browser side. With fragment, code and state >> are not sent to client application, so it's one less possibility how >> they can be compromised. >> >> - Added support for "nonce" check in keycloak.js. It's useful for some >> kind of attacks and specs wants it for implicit flow. >> >> Possible pending work: >> - Fix integration-arquillian as PR is failing now >> >> - Possibly add server-side support for "at_hash" to IDToken ? Not sure >> if at_hash support has any real advantages for us, however specs >> requires that and even Google supports it. >> >> - Add docs and tests. Not sure about adding separate example? Until now, >> I've tested with existing js-console example and added support here for >> easily set init parameters to keycloak.init, so people can use different >> flow or responseMode here. >> > > Docs is enough. I don't think there's any need for examples. > > >> >> WDYT? >> >> Marek >> >> [1] >> >> http://openid.net/specs/oauth-v2-multiple-response-types-1_0.html#Combinations >> >> [2] >> >> http://openid.net/specs/oauth-v2-multiple-response-types-1_0.html#ResponseModes >> >> [3] http://openid.net/specs/oauth-v2-form-post-response-mode-1_0.html >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/d5593e04/attachment-0001.html From velias at redhat.com Fri Nov 27 06:16:32 2015 From: velias at redhat.com (Vlastimil Elias) Date: Fri, 27 Nov 2015 12:16:32 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: References: <5658210D.1070301@redhat.com> Message-ID: <56583B90.3090904@redhat.com> Hi, On 27.11.2015 11:45, Stian Thorgersen wrote: > > > On 27 November 2015 at 10:23, Vlastimil Elias > wrote: > > Hi, > > I have two proposals for cleanup of 'Change password' screen in > Account > app based on my experience with it: > > 1. remove Cancel button - it has no any meaning on this > screen/form, it > only reshowns form with empty fields. And also there is a bug, > "Password" field is hidden when it is used, which makes whole form > unusable. > > > +1 OK, I'm going to create JIRA and provide PR for this. > > > > 2. remove validation of current password (remove "Password" > field). Two > reasons for this: > - security impact of this check is small. If attacker is able to > compromise Account app then he can always change email and then use > "Forgot password" feature to change password > - user created over Identity Provider do not know old password > (because it is not set) so he is not able to set password using > this screen > After we implement support for reauthentication (KEYCLOAK-2076) > then we > should set some reasonable reauth timeout for Account app instead, > this > will make it more secure at all. > > > -1 Reset password over email may not be enabled at all. We already > allow setting password for IdPs login without requiring the existing > password. Fair enough > > +1 To suggestion from Thomas - we should ask for password when > updating email at least when recover password over email is enabled. Makes sense, but what to do if user has no password set at this point? Don't ask him, or reauthenticate him by other available mechanism? And there are also other "dangerous" operations in Account app. Eg. attacker who gains access to it can disable OTP without any recheck. This should be protected too. I believe whole Account app should be correctly protected by reauthentications. Question is if implement it somehow specifically in the app, or resolve this generally as part of KEYCLOAK-2076. Vlastimil > > It seems to be common practice to ask for current password when > updating the existing password. > > > > If you agree then I can create JIRA issue for this and provide PR. > > Vlastimil > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -- Vlastimil Elias Principal Software Engineer Developer Portal Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/64748b85/attachment.html From sthorger at redhat.com Fri Nov 27 06:22:18 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 12:22:18 +0100 Subject: [keycloak-dev] [keycloak-user] How to implement long user sso sessions with reauthentication for important actions? In-Reply-To: <56582AEE.4020508@redhat.com> References: <5644910D.9020100@redhat.com> <564498DE.6010805@redhat.com> <56582AEE.4020508@redhat.com> Message-ID: Initially when you proposed this feature I underestimated how complicated it would be as well. Reading the discussion on the issue you created I've got a few comments: * We need a new flow - users could want to require re-auth with password, or with totp, or with both, etc.. * There's probably quite a few corner cases to cover - you've already mentioned a few It's a very nice to have feature, but I doubt have time to work on it for 1.8. Is it something we can post-pone to 2.x or do you really need it now? On 27 November 2015 at 11:05, Vlastimil Elias wrote: > Hi, > > moving this discussion to the devel forum as it is about the feture > development now. > > Toplevel issue I created for this feature is > https://issues.jboss.org/browse/KEYCLOAK-2076 > > I added some notes and thoughts from my investigation as a comment to the > KEYCLOAK-2076, there are some open questions how to implement it. > > Originally I though I should be able to implement reauth support and > provide PR. > But I must say I'm not sure now if I'm able to implement it, looks like it > is a bit more complicated than I originally expected, so probably some > Keycloak core developer should do it. > But if you think you will not have resources to do it in 1.8 then I can > try it (with your support), as I believe it is a very important feature, > and we really want use it. > > Cheers > > Vlastimil > > > > > On 12.11.2015 14:50, Stian Thorgersen wrote: > > > > On 12 November 2015 at 14:49, Vlastimil Elias < > velias at redhat.com> wrote: > >> Thanks for quick reply Stian. >> >> I'm going to create JIRAs for all these things. I can volunter to >> implement some parts of this. >> >> For the last one, it should be probably cool to have "reauthenticate >> timeout" setting available in client section for every client (not only >> internal admin console and account management). It should allow simple >> implementation of "long user sso session" scheme even in environments where >> some clients can't be updated to set max_age on protocol level. >> > > Yep, that makes sense > > >> >> Vl. >> >> >> On 12.11.2015 14:39, Stian Thorgersen wrote: >> >> >> >> On 12 November 2015 at 14:15, Vlastimil Elias < >> velias at redhat.com> wrote: >> >>> Hi, >>> >>> I'd like to use long session authentication mechanism known from many >>> sites like google. facebook, linked in etc. >>> It is about really long user SSO sessions (eg. weeks or even months) >>> with reauthentication for important actions when last authentication >>> timestamp is older than some limit. >>> >>> Is this somehow possible with current Keycloak server and Keycloak >>> adapters? >>> >>> I see few subquestions in this problem for our use: >>> >>> ***** >>> open-id connect protocol defines few auth request parameters to support >>> this use case, mainly max_age or prompt=login. Are they correctly >>> implemented in Keycloak server? >>> >> >> We don't have support for max_age and we only support prompt=none so >> these would have to be added >> >> >>> >>> >>> ***** >>> Wildfly/EAP adapter - is it possible and is there some example how to >>> use "reauth if auth is older than 30min" action in Java app secured by >>> this adapter? Or is info about last auth timestamp somehow available in >>> the app? >>> >> >> We don't set auth_time claim ATM so answer is no >> >> >>> >>> >>> ***** >>> Keycloak user account application itself - it is part of the Keycloak >>> server, but it contains sensitive actions which typically require >>> reathentication in this long session scheme (password change, email >>> change, ...). Is it somehow possible to configure Keycloak to force >>> timeout reauth for this app? >>> >> >> Not at the moment - but if we add what you want it would also make sense >> to add that. Would need to be configurable through the admin console. Would >> also be nice to have the same for the admin console itself. >> >> >>> >>> Thanks in advance >>> >>> Vl. >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >>> >>> _______________________________________________ >>> keycloak-user mailing list >>> keycloak-user at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-user >>> >> >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/31a1bca7/attachment.html From sthorger at redhat.com Fri Nov 27 06:28:41 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 12:28:41 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: <56583B90.3090904@redhat.com> References: <5658210D.1070301@redhat.com> <56583B90.3090904@redhat.com> Message-ID: On 27 November 2015 at 12:16, Vlastimil Elias wrote: > Hi, > > On 27.11.2015 11:45, Stian Thorgersen wrote: > > > > On 27 November 2015 at 10:23, Vlastimil Elias < > velias at redhat.com> wrote: > >> Hi, >> >> I have two proposals for cleanup of 'Change password' screen in Account >> app based on my experience with it: >> >> 1. remove Cancel button - it has no any meaning on this screen/form, it >> only reshowns form with empty fields. And also there is a bug, >> "Password" field is hidden when it is used, which makes whole form >> unusable. >> > > +1 > > > OK, I'm going to create JIRA and provide PR for this. > > > >> >> 2. remove validation of current password (remove "Password" field). Two >> reasons for this: >> - security impact of this check is small. If attacker is able to >> compromise Account app then he can always change email and then use >> "Forgot password" feature to change password >> - user created over Identity Provider do not know old password >> (because it is not set) so he is not able to set password using this >> screen >> After we implement support for reauthentication (KEYCLOAK-2076) then we >> should set some reasonable reauth timeout for Account app instead, this >> will make it more secure at all. >> > > -1 Reset password over email may not be enabled at all. We already allow > setting password for IdPs login without requiring the existing password. > > Fair enough > > > +1 To suggestion from Thomas - we should ask for password when updating > email at least when recover password over email is enabled. > > > Makes sense, but what to do if user has no password set at this point? > Don't ask him, or reauthenticate him by other available mechanism? > > And there are also other "dangerous" operations in Account app. Eg. > attacker who gains access to it can disable OTP without any recheck. This > should be protected too. > I believe whole Account app should be correctly protected by > reauthentications. Question is if implement it somehow specifically in the > app, or resolve this generally as part of KEYCLOAK-2076. > Yep, you're right there's other dangerous operations as well. Re-authentication sounds like the correct approach, rather than requesting the password again. We could have two configurable timeouts in account management console. One that requires re-auth for anything (~30 min default), and another that requires re-auth for sensitive operations (~5 min). One thing with re-auth is that if it happens when a user submits a form, you then someone have to also remember the values of the form. Currently account management is stateless. > > > Vlastimil > > > It seems to be common practice to ask for current password when updating > the existing password. > > >> >> If you agree then I can create JIRA issue for this and provide PR. >> >> Vlastimil >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/6599ae13/attachment-0001.html From velias at redhat.com Fri Nov 27 07:22:05 2015 From: velias at redhat.com (Vlastimil Elias) Date: Fri, 27 Nov 2015 13:22:05 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: References: <5658210D.1070301@redhat.com> <56583B90.3090904@redhat.com> Message-ID: <56584AED.4040803@redhat.com> On 27.11.2015 12:28, Stian Thorgersen wrote: > > > On 27 November 2015 at 12:16, Vlastimil Elias > wrote: > > Hi, > > On 27.11.2015 11:45, Stian Thorgersen wrote: >> >> >> On 27 November 2015 at 10:23, Vlastimil Elias > > wrote: >> >> Hi, >> >> I have two proposals for cleanup of 'Change password' screen >> in Account >> app based on my experience with it: >> >> 1. remove Cancel button - it has no any meaning on this >> screen/form, it >> only reshowns form with empty fields. And also there is a bug, >> "Password" field is hidden when it is used, which makes whole >> form unusable. >> >> >> +1 > > OK, I'm going to create JIRA and provide PR for this. > >> >> >> >> 2. remove validation of current password (remove "Password" >> field). Two >> reasons for this: >> - security impact of this check is small. If attacker is >> able to >> compromise Account app then he can always change email and >> then use >> "Forgot password" feature to change password >> - user created over Identity Provider do not know old password >> (because it is not set) so he is not able to set password >> using this screen >> After we implement support for reauthentication >> (KEYCLOAK-2076) then we >> should set some reasonable reauth timeout for Account app >> instead, this >> will make it more secure at all. >> >> >> -1 Reset password over email may not be enabled at all. We >> already allow setting password for IdPs login without requiring >> the existing password. > Fair enough >> >> +1 To suggestion from Thomas - we should ask for password when >> updating email at least when recover password over email is enabled. > > Makes sense, but what to do if user has no password set at this > point? Don't ask him, or reauthenticate him by other available > mechanism? > > And there are also other "dangerous" operations in Account app. > Eg. attacker who gains access to it can disable OTP without any > recheck. This should be protected too. > I believe whole Account app should be correctly protected by > reauthentications. Question is if implement it somehow > specifically in the app, or resolve this generally as part of > KEYCLOAK-2076. > > > Yep, you're right there's other dangerous operations as well. > Re-authentication sounds like the correct approach, rather than > requesting the password again. We could have two configurable timeouts > in account management console. One that requires re-auth for anything > (~30 min default), and another that requires re-auth for sensitive > operations (~5 min). > > One thing with re-auth is that if it happens when a user submits a > form, you then someone have to also remember the values of the form. > Currently account management is stateless. Yep, implementing reauthentication schemes correctly in client apps may be tricky, mainly for POST requests (and even worse posts with file upload) and AJAX call handlers etc. Beside Keycloak Account app we should try to provide reasonable support in Keycloak adapters, but it may be tricky as related Java EE specifications (JAAS subsystems, auth related things in Java Servlet and EJB spec ) do not count with these more complicated schemes too much. And I also thought if reauthentication scheme should be somehow implemented on REST API levels also. Eg. some REST endpoint operation will define that it needs reauth with some timeout. If client app call it with Bearer token then it should be able to tests last auth timestamp (which should be possible as it may be in the token as claim) and propagate back info that it needs reauth (how to map this to HTTP?). Support for this should be added to the framework like RestEasy then. Vl. > > > > > Vlastimil > >> >> It seems to be common practice to ask for current password when >> updating the existing password. >> >> >> >> If you agree then I can create JIRA issue for this and >> provide PR. >> >> Vlastimil >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> >> >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -- Vlastimil Elias Principal Software Engineer Developer Portal Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/f8aab688/attachment.html From sthorger at redhat.com Fri Nov 27 07:25:18 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 13:25:18 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: <56584AED.4040803@redhat.com> References: <5658210D.1070301@redhat.com> <56583B90.3090904@redhat.com> <56584AED.4040803@redhat.com> Message-ID: Hmm.. re-auth request for rest endpoints is even more tricky. I think a rest endpoint would just have to return a 401? Then it's up to the client to figure out that it needs to re-authenticate. On 27 November 2015 at 13:22, Vlastimil Elias wrote: > > > On 27.11.2015 12:28, Stian Thorgersen wrote: > > > > On 27 November 2015 at 12:16, Vlastimil Elias < > velias at redhat.com> wrote: > >> Hi, >> >> On 27.11.2015 11:45, Stian Thorgersen wrote: >> >> >> >> On 27 November 2015 at 10:23, Vlastimil Elias < >> velias at redhat.com> wrote: >> >>> Hi, >>> >>> I have two proposals for cleanup of 'Change password' screen in Account >>> app based on my experience with it: >>> >>> 1. remove Cancel button - it has no any meaning on this screen/form, it >>> only reshowns form with empty fields. And also there is a bug, >>> "Password" field is hidden when it is used, which makes whole form >>> unusable. >>> >> >> +1 >> >> >> OK, I'm going to create JIRA and provide PR for this. >> >> >> >>> >>> 2. remove validation of current password (remove "Password" field). Two >>> reasons for this: >>> - security impact of this check is small. If attacker is able to >>> compromise Account app then he can always change email and then use >>> "Forgot password" feature to change password >>> - user created over Identity Provider do not know old password >>> (because it is not set) so he is not able to set password using this >>> screen >>> After we implement support for reauthentication (KEYCLOAK-2076) then we >>> should set some reasonable reauth timeout for Account app instead, this >>> will make it more secure at all. >>> >> >> -1 Reset password over email may not be enabled at all. We already allow >> setting password for IdPs login without requiring the existing password. >> >> Fair enough >> >> >> +1 To suggestion from Thomas - we should ask for password when updating >> email at least when recover password over email is enabled. >> >> >> Makes sense, but what to do if user has no password set at this point? >> Don't ask him, or reauthenticate him by other available mechanism? >> >> And there are also other "dangerous" operations in Account app. Eg. >> attacker who gains access to it can disable OTP without any recheck. This >> should be protected too. >> I believe whole Account app should be correctly protected by >> reauthentications. Question is if implement it somehow specifically in the >> app, or resolve this generally as part of KEYCLOAK-2076. >> > > Yep, you're right there's other dangerous operations as well. > Re-authentication sounds like the correct approach, rather than requesting > the password again. We could have two configurable timeouts in account > management console. One that requires re-auth for anything (~30 min > default), and another that requires re-auth for sensitive operations (~5 > min). > > One thing with re-auth is that if it happens when a user submits a form, > you then someone have to also remember the values of the form. Currently > account management is stateless. > > > Yep, implementing reauthentication schemes correctly in client apps may be > tricky, mainly for POST requests (and even worse posts with file upload) > and AJAX call handlers etc. > Beside Keycloak Account app we should try to provide reasonable support in > Keycloak adapters, but it may be tricky as related Java EE specifications > (JAAS subsystems, auth related things in Java Servlet and EJB spec ) do not > count with these more complicated schemes too much. > > And I also thought if reauthentication scheme should be somehow > implemented on REST API levels also. > Eg. some REST endpoint operation will define that it needs reauth with > some timeout. If client app call it with Bearer token then it should be > able to tests last auth timestamp (which should be possible as it may be in > the token as claim) and propagate back info that it needs reauth (how to > map this to HTTP?). Support for this should be added to the framework like > RestEasy then. > > Vl. > > > >> >> >> Vlastimil >> >> >> It seems to be common practice to ask for current password when updating >> the existing password. >> >> >>> >>> If you agree then I can create JIRA issue for this and provide PR. >>> >>> Vlastimil >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/2e998afe/attachment-0001.html From velias at redhat.com Fri Nov 27 07:30:15 2015 From: velias at redhat.com (Vlastimil Elias) Date: Fri, 27 Nov 2015 13:30:15 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: References: <5658210D.1070301@redhat.com> <56583B90.3090904@redhat.com> <56584AED.4040803@redhat.com> Message-ID: <56584CD7.3090407@redhat.com> On 27.11.2015 13:25, Stian Thorgersen wrote: > Hmm.. re-auth request for rest endpoints is even more tricky. I think > a rest endpoint would just have to return a 401? Then it's up to the > client to figure out that it needs to re-authenticate. Yep, 401 matches best probably, but at least hint in some response header should be cool, probably in WWW-Authenticate as it is mandatory for 401 ;-) But this is the future, proper Keycloak server support is necessary first ;-) Vl. > > On 27 November 2015 at 13:22, Vlastimil Elias > wrote: > > > > On 27.11.2015 12:28, Stian Thorgersen wrote: >> >> >> On 27 November 2015 at 12:16, Vlastimil Elias > > wrote: >> >> Hi, >> >> On 27.11.2015 11:45, Stian Thorgersen wrote: >>> >>> >>> On 27 November 2015 at 10:23, Vlastimil Elias >>> > wrote: >>> >>> Hi, >>> >>> I have two proposals for cleanup of 'Change password' >>> screen in Account >>> app based on my experience with it: >>> >>> 1. remove Cancel button - it has no any meaning on this >>> screen/form, it >>> only reshowns form with empty fields. And also there is >>> a bug, >>> "Password" field is hidden when it is used, which makes >>> whole form unusable. >>> >>> >>> +1 >> >> OK, I'm going to create JIRA and provide PR for this. >> >>> >>> >>> >>> 2. remove validation of current password (remove >>> "Password" field). Two >>> reasons for this: >>> - security impact of this check is small. If attacker >>> is able to >>> compromise Account app then he can always change email >>> and then use >>> "Forgot password" feature to change password >>> - user created over Identity Provider do not know old >>> password >>> (because it is not set) so he is not able to set >>> password using this screen >>> After we implement support for reauthentication >>> (KEYCLOAK-2076) then we >>> should set some reasonable reauth timeout for Account >>> app instead, this >>> will make it more secure at all. >>> >>> >>> -1 Reset password over email may not be enabled at all. We >>> already allow setting password for IdPs login without >>> requiring the existing password. >> Fair enough >>> >>> +1 To suggestion from Thomas - we should ask for password >>> when updating email at least when recover password over >>> email is enabled. >> >> Makes sense, but what to do if user has no password set at >> this point? Don't ask him, or reauthenticate him by other >> available mechanism? >> >> And there are also other "dangerous" operations in Account >> app. Eg. attacker who gains access to it can disable OTP >> without any recheck. This should be protected too. >> I believe whole Account app should be correctly protected by >> reauthentications. Question is if implement it somehow >> specifically in the app, or resolve this generally as part of >> KEYCLOAK-2076. >> >> >> Yep, you're right there's other dangerous operations as well. >> Re-authentication sounds like the correct approach, rather than >> requesting the password again. We could have two configurable >> timeouts in account management console. One that requires re-auth >> for anything (~30 min default), and another that requires re-auth >> for sensitive operations (~5 min). >> >> One thing with re-auth is that if it happens when a user submits >> a form, you then someone have to also remember the values of the >> form. Currently account management is stateless. > > Yep, implementing reauthentication schemes correctly in client > apps may be tricky, mainly for POST requests (and even worse posts > with file upload) and AJAX call handlers etc. > Beside Keycloak Account app we should try to provide reasonable > support in Keycloak adapters, but it may be tricky as related Java > EE specifications (JAAS subsystems, auth related things in Java > Servlet and EJB spec ) do not count with these more complicated > schemes too much. > > And I also thought if reauthentication scheme should be somehow > implemented on REST API levels also. > Eg. some REST endpoint operation will define that it needs reauth > with some timeout. If client app call it with Bearer token then it > should be able to tests last auth timestamp (which should be > possible as it may be in the token as claim) and propagate back > info that it needs reauth (how to map this to HTTP?). Support for > this should be added to the framework like RestEasy then. > > Vl. > >> >> >> >> >> Vlastimil >> >>> >>> It seems to be common practice to ask for current password >>> when updating the existing password. >>> >>> >>> >>> If you agree then I can create JIRA issue for this and >>> provide PR. >>> >>> Vlastimil >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -- Vlastimil Elias Principal Software Engineer Developer Portal Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/afd5fb4c/attachment.html From velias at redhat.com Fri Nov 27 07:36:44 2015 From: velias at redhat.com (Vlastimil Elias) Date: Fri, 27 Nov 2015 13:36:44 +0100 Subject: [keycloak-dev] [keycloak-user] How to implement long user sso sessions with reauthentication for important actions? In-Reply-To: References: <5644910D.9020100@redhat.com> <564498DE.6010805@redhat.com> <56582AEE.4020508@redhat.com> Message-ID: <56584E5C.1060802@redhat.com> On 27.11.2015 12:22, Stian Thorgersen wrote: > Initially when you proposed this feature I underestimated how > complicated it would be as well. Reading the discussion on the issue > you created I've got a few comments: > > * We need a new flow - users could want to require re-auth with > password, or with totp, or with both, etc.. Agree, but at least first step must be same with Browser flow - check of cookie, and then redirect to another flow if necessary. Not sure if current flow engine supports this. > * There's probably quite a few corner cases to cover - you've already > mentioned a few > > It's a very nice to have feature, but I doubt have time to work on it > for 1.8. > > Is it something we can post-pone to 2.x or do you really need it now? We do not need it just now, but should be cool to have it in some reasonable time (half of year?). Not sure when you plan to start work on 2.x, probably sometime around half of next year? As it is really complex then we probably should break it into smaller chunks and implement it per partes over the releases. I can imagine first version with basic flow support with Form password auth only, then add other parts like support for Identity Brokers, OTP, Kerberos, required actions etc. Vl. > > On 27 November 2015 at 11:05, Vlastimil Elias > wrote: > > Hi, > > moving this discussion to the devel forum as it is about the > feture development now. > > Toplevel issue I created for this feature is > https://issues.jboss.org/browse/KEYCLOAK-2076 > > I added some notes and thoughts from my investigation as a comment > to the KEYCLOAK-2076, there are some open questions how to > implement it. > > Originally I though I should be able to implement reauth support > and provide PR. > But I must say I'm not sure now if I'm able to implement it, looks > like it is a bit more complicated than I originally expected, so > probably some Keycloak core developer should do it. > But if you think you will not have resources to do it in 1.8 then > I can try it (with your support), as I believe it is a very > important feature, and we really want use it. > > Cheers > > Vlastimil > > > > > On 12.11.2015 14:50, Stian Thorgersen wrote: >> >> >> On 12 November 2015 at 14:49, Vlastimil Elias > > wrote: >> >> Thanks for quick reply Stian. >> >> I'm going to create JIRAs for all these things. I can >> volunter to implement some parts of this. >> >> For the last one, it should be probably cool to have >> "reauthenticate timeout" setting available in client section >> for every client (not only internal admin console and account >> management). It should allow simple implementation of "long >> user sso session" scheme even in environments where some >> clients can't be updated to set max_age on protocol level. >> >> >> Yep, that makes sense >> >> >> >> Vl. >> >> >> On 12.11.2015 14:39, Stian Thorgersen wrote: >>> >>> >>> On 12 November 2015 at 14:15, Vlastimil Elias >>> > wrote: >>> >>> Hi, >>> >>> I'd like to use long session authentication mechanism >>> known from many >>> sites like google. facebook, linked in etc. >>> It is about really long user SSO sessions (eg. weeks or >>> even months) >>> with reauthentication for important actions when last >>> authentication >>> timestamp is older than some limit. >>> >>> Is this somehow possible with current Keycloak server >>> and Keycloak adapters? >>> >>> I see few subquestions in this problem for our use: >>> >>> ***** >>> open-id connect protocol defines few auth request >>> parameters to support >>> this use case, mainly max_age or prompt=login. Are they >>> correctly >>> implemented in Keycloak server? >>> >>> >>> We don't have support for max_age and we only >>> support prompt=none so these would have to be added >>> >>> >>> >>> >>> ***** >>> Wildfly/EAP adapter - is it possible and is there some >>> example how to >>> use "reauth if auth is older than 30min" action in Java >>> app secured by >>> this adapter? Or is info about last auth timestamp >>> somehow available in >>> the app? >>> >>> >>> We don't set auth_time claim ATM so answer is no >>> >>> >>> >>> >>> ***** >>> Keycloak user account application itself - it is part of >>> the Keycloak >>> server, but it contains sensitive actions which >>> typically require >>> reathentication in this long session scheme (password >>> change, email >>> change, ...). Is it somehow possible to configure >>> Keycloak to force >>> timeout reauth for this app? >>> >>> >>> Not at the moment - but if we add what you want it would >>> also make sense to add that. Would need to be configurable >>> through the admin console. Would also be nice to have the >>> same for the admin console itself. >>> >>> >>> >>> Thanks in advance >>> >>> Vl. >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >>> >>> _______________________________________________ >>> keycloak-user mailing list >>> keycloak-user at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-user >>> >>> >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -- Vlastimil Elias Principal Software Engineer Developer Portal Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/0c684364/attachment-0001.html From sthorger at redhat.com Fri Nov 27 07:46:03 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 13:46:03 +0100 Subject: [keycloak-dev] [keycloak-user] How to implement long user sso sessions with reauthentication for important actions? In-Reply-To: <56584E5C.1060802@redhat.com> References: <5644910D.9020100@redhat.com> <564498DE.6010805@redhat.com> <56582AEE.4020508@redhat.com> <56584E5C.1060802@redhat.com> Message-ID: On 27 November 2015 at 13:36, Vlastimil Elias wrote: > > > On 27.11.2015 12:22, Stian Thorgersen wrote: > > Initially when you proposed this feature I underestimated how complicated > it would be as well. Reading the discussion on the issue you created I've > got a few comments: > > * We need a new flow - users could want to require re-auth with password, > or with totp, or with both, etc.. > > Agree, but at least first step must be same with Browser flow - check of > cookie, and then redirect to another flow if necessary. Not sure if current > flow engine supports this. > Current browser flow would just automatically log-in as the cookie is there > > > * There's probably quite a few corner cases to cover - you've already > mentioned a few > > It's a very nice to have feature, but I doubt have time to work on it for > 1.8. > > Is it something we can post-pone to 2.x or do you really need it now? > > > We do not need it just now, but should be cool to have it in some > reasonable time (half of year?). > Not sure when you plan to start work on 2.x, probably sometime around half > of next year? > > As it is really complex then we probably should break it into smaller > chunks and implement it per partes over the releases. I can imagine first > version with basic flow support with Form password auth only, then add > other parts like support for Identity Brokers, OTP, Kerberos, required > actions etc. > > Vl. > > > > On 27 November 2015 at 11:05, Vlastimil Elias wrote: > >> Hi, >> >> moving this discussion to the devel forum as it is about the feture >> development now. >> >> Toplevel issue I created for this feature is >> >> https://issues.jboss.org/browse/KEYCLOAK-2076 >> >> I added some notes and thoughts from my investigation as a comment to the >> KEYCLOAK-2076, there are some open questions how to implement it. >> >> Originally I though I should be able to implement reauth support and >> provide PR. >> But I must say I'm not sure now if I'm able to implement it, looks like >> it is a bit more complicated than I originally expected, so probably some >> Keycloak core developer should do it. >> But if you think you will not have resources to do it in 1.8 then I can >> try it (with your support), as I believe it is a very important feature, >> and we really want use it. >> >> Cheers >> >> Vlastimil >> >> >> >> >> On 12.11.2015 14:50, Stian Thorgersen wrote: >> >> >> >> On 12 November 2015 at 14:49, Vlastimil Elias < >> velias at redhat.com> wrote: >> >>> Thanks for quick reply Stian. >>> >>> I'm going to create JIRAs for all these things. I can volunter to >>> implement some parts of this. >>> >>> For the last one, it should be probably cool to have "reauthenticate >>> timeout" setting available in client section for every client (not only >>> internal admin console and account management). It should allow simple >>> implementation of "long user sso session" scheme even in environments where >>> some clients can't be updated to set max_age on protocol level. >>> >> >> Yep, that makes sense >> >> >>> >>> Vl. >>> >>> >>> On 12.11.2015 14:39, Stian Thorgersen wrote: >>> >>> >>> >>> On 12 November 2015 at 14:15, Vlastimil Elias < >>> velias at redhat.com> wrote: >>> >>>> Hi, >>>> >>>> I'd like to use long session authentication mechanism known from many >>>> sites like google. facebook, linked in etc. >>>> It is about really long user SSO sessions (eg. weeks or even months) >>>> with reauthentication for important actions when last authentication >>>> timestamp is older than some limit. >>>> >>>> Is this somehow possible with current Keycloak server and Keycloak >>>> adapters? >>>> >>>> I see few subquestions in this problem for our use: >>>> >>>> ***** >>>> open-id connect protocol defines few auth request parameters to support >>>> this use case, mainly max_age or prompt=login. Are they correctly >>>> implemented in Keycloak server? >>>> >>> >>> We don't have support for max_age and we only support prompt=none so >>> these would have to be added >>> >>> >>>> >>>> >>>> ***** >>>> Wildfly/EAP adapter - is it possible and is there some example how to >>>> use "reauth if auth is older than 30min" action in Java app secured by >>>> this adapter? Or is info about last auth timestamp somehow available in >>>> the app? >>>> >>> >>> We don't set auth_time claim ATM so answer is no >>> >>> >>>> >>>> >>>> ***** >>>> Keycloak user account application itself - it is part of the Keycloak >>>> server, but it contains sensitive actions which typically require >>>> reathentication in this long session scheme (password change, email >>>> change, ...). Is it somehow possible to configure Keycloak to force >>>> timeout reauth for this app? >>>> >>> >>> Not at the moment - but if we add what you want it would also make sense >>> to add that. Would need to be configurable through the admin console. Would >>> also be nice to have the same for the admin console itself. >>> >>> >>>> >>>> Thanks in advance >>>> >>>> Vl. >>>> >>>> -- >>>> Vlastimil Elias >>>> Principal Software Engineer >>>> Developer Portal Engineering Team >>>> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-user mailing list >>>> keycloak-user at lists.jboss.org >>>> >>>> https://lists.jboss.org/mailman/listinfo/keycloak-user >>>> >>> >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/1cb44515/attachment.html From velias at redhat.com Fri Nov 27 07:51:28 2015 From: velias at redhat.com (Vlastimil Elias) Date: Fri, 27 Nov 2015 13:51:28 +0100 Subject: [keycloak-dev] [keycloak-user] How to implement long user sso sessions with reauthentication for important actions? In-Reply-To: References: <5644910D.9020100@redhat.com> <564498DE.6010805@redhat.com> <56582AEE.4020508@redhat.com> <56584E5C.1060802@redhat.com> Message-ID: <565851D0.40008@redhat.com> On 27.11.2015 13:46, Stian Thorgersen wrote: > > > On 27 November 2015 at 13:36, Vlastimil Elias > wrote: > > > > On 27.11.2015 12:22, Stian Thorgersen wrote: >> Initially when you proposed this feature I underestimated how >> complicated it would be as well. Reading the discussion on the >> issue you created I've got a few comments: >> >> * We need a new flow - users could want to require re-auth with >> password, or with totp, or with both, etc.. > Agree, but at least first step must be same with Browser flow - > check of cookie, and then redirect to another flow if necessary. > Not sure if current flow engine supports this. > > > Current browser flow would just automatically log-in as the cookie is > there yep, but new Reauthentication flow must also check if user is logged in currently (over cookie). If not logged in then it should go to current Browser flow to login him. If logged in then it must continue in Reathentication flow. So check "is user currently logged in?" should be same for both flows. Vl > > > > >> * There's probably quite a few corner cases to cover - you've >> already mentioned a few >> >> It's a very nice to have feature, but I doubt have time to work >> on it for 1.8. >> >> Is it something we can post-pone to 2.x or do you really need it now? > > We do not need it just now, but should be cool to have it in some > reasonable time (half of year?). > Not sure when you plan to start work on 2.x, probably sometime > around half of next year? > > As it is really complex then we probably should break it into > smaller chunks and implement it per partes over the releases. I > can imagine first version with basic flow support with Form > password auth only, then add other parts like support for Identity > Brokers, OTP, Kerberos, required actions etc. > > Vl. > > >> >> On 27 November 2015 at 11:05, Vlastimil Elias > > wrote: >> >> Hi, >> >> moving this discussion to the devel forum as it is about the >> feture development now. >> >> Toplevel issue I created for this feature is >> https://issues.jboss.org/browse/KEYCLOAK-2076 >> >> I added some notes and thoughts from my investigation as a >> comment to the KEYCLOAK-2076, there are some open questions >> how to implement it. >> >> Originally I though I should be able to implement reauth >> support and provide PR. >> But I must say I'm not sure now if I'm able to implement it, >> looks like it is a bit more complicated than I originally >> expected, so probably some Keycloak core developer should do it. >> But if you think you will not have resources to do it in 1.8 >> then I can try it (with your support), as I believe it is a >> very important feature, and we really want use it. >> >> Cheers >> >> Vlastimil >> >> >> >> >> On 12.11.2015 14:50, Stian Thorgersen wrote: >>> >>> >>> On 12 November 2015 at 14:49, Vlastimil Elias >>> > wrote: >>> >>> Thanks for quick reply Stian. >>> >>> I'm going to create JIRAs for all these things. I can >>> volunter to implement some parts of this. >>> >>> For the last one, it should be probably cool to have >>> "reauthenticate timeout" setting available in client >>> section for every client (not only internal admin >>> console and account management). It should allow simple >>> implementation of "long user sso session" scheme even in >>> environments where some clients can't be updated to set >>> max_age on protocol level. >>> >>> >>> Yep, that makes sense >>> >>> >>> >>> Vl. >>> >>> >>> On 12.11.2015 14:39, Stian Thorgersen wrote: >>>> >>>> >>>> On 12 November 2015 at 14:15, Vlastimil Elias >>>> > wrote: >>>> >>>> Hi, >>>> >>>> I'd like to use long session authentication >>>> mechanism known from many >>>> sites like google. facebook, linked in etc. >>>> It is about really long user SSO sessions (eg. >>>> weeks or even months) >>>> with reauthentication for important actions when >>>> last authentication >>>> timestamp is older than some limit. >>>> >>>> Is this somehow possible with current Keycloak >>>> server and Keycloak adapters? >>>> >>>> I see few subquestions in this problem for our use: >>>> >>>> ***** >>>> open-id connect protocol defines few auth request >>>> parameters to support >>>> this use case, mainly max_age or prompt=login. Are >>>> they correctly >>>> implemented in Keycloak server? >>>> >>>> >>>> We don't have support for max_age and we only >>>> support prompt=none so these would have to be added >>>> >>>> >>>> >>>> >>>> ***** >>>> Wildfly/EAP adapter - is it possible and is there >>>> some example how to >>>> use "reauth if auth is older than 30min" action in >>>> Java app secured by >>>> this adapter? Or is info about last auth timestamp >>>> somehow available in >>>> the app? >>>> >>>> >>>> We don't set auth_time claim ATM so answer is no >>>> >>>> >>>> >>>> >>>> ***** >>>> Keycloak user account application itself - it is >>>> part of the Keycloak >>>> server, but it contains sensitive actions which >>>> typically require >>>> reathentication in this long session scheme >>>> (password change, email >>>> change, ...). Is it somehow possible to configure >>>> Keycloak to force >>>> timeout reauth for this app? >>>> >>>> >>>> Not at the moment - but if we add what you want it >>>> would also make sense to add that. Would need to be >>>> configurable through the admin console. Would also be >>>> nice to have the same for the admin console itself. >>>> >>>> >>>> >>>> Thanks in advance >>>> >>>> Vl. >>>> >>>> -- >>>> Vlastimil Elias >>>> Principal Software Engineer >>>> Developer Portal Engineering Team >>>> >>>> >>>> >>>> _______________________________________________ >>>> keycloak-user mailing list >>>> keycloak-user at lists.jboss.org >>>> >>>> https://lists.jboss.org/mailman/listinfo/keycloak-user >>>> >>>> >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -- Vlastimil Elias Principal Software Engineer Developer Portal Engineering Team -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/80209b27/attachment-0001.html From sthorger at redhat.com Fri Nov 27 07:57:47 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 13:57:47 +0100 Subject: [keycloak-dev] [keycloak-user] How to implement long user sso sessions with reauthentication for important actions? In-Reply-To: <565851D0.40008@redhat.com> References: <5644910D.9020100@redhat.com> <564498DE.6010805@redhat.com> <56582AEE.4020508@redhat.com> <56584E5C.1060802@redhat.com> <565851D0.40008@redhat.com> Message-ID: Current flow checks if cookie is there and if it is it's all happy. We'd need a different flow that can require the re-auth to take place even if cookie is there. Also, with re-auth flow user should be able to cancel or just navigate away and continue using the session as if nothing has happend (with the other apps that's happy with the old session that is) On 27 November 2015 at 13:51, Vlastimil Elias wrote: > > > On 27.11.2015 13:46, Stian Thorgersen wrote: > > > > On 27 November 2015 at 13:36, Vlastimil Elias < > velias at redhat.com> wrote: > >> >> >> On 27.11.2015 12:22, Stian Thorgersen wrote: >> >> Initially when you proposed this feature I underestimated how complicated >> it would be as well. Reading the discussion on the issue you created I've >> got a few comments: >> >> * We need a new flow - users could want to require re-auth with password, >> or with totp, or with both, etc.. >> >> Agree, but at least first step must be same with Browser flow - check of >> cookie, and then redirect to another flow if necessary. Not sure if current >> flow engine supports this. >> > > Current browser flow would just automatically log-in as the cookie is there > > > yep, but new Reauthentication flow must also check if user is logged in > currently (over cookie). If not logged in then it should go to current > Browser flow to login him. If logged in then it must continue in > Reathentication flow. So check "is user currently logged in?" should be > same for both flows. > > Vl > > > > >> >> >> * There's probably quite a few corner cases to cover - you've already >> mentioned a few >> >> It's a very nice to have feature, but I doubt have time to work on it for >> 1.8. >> >> Is it something we can post-pone to 2.x or do you really need it now? >> >> >> We do not need it just now, but should be cool to have it in some >> reasonable time (half of year?). >> Not sure when you plan to start work on 2.x, probably sometime around >> half of next year? >> >> As it is really complex then we probably should break it into smaller >> chunks and implement it per partes over the releases. I can imagine first >> version with basic flow support with Form password auth only, then add >> other parts like support for Identity Brokers, OTP, Kerberos, required >> actions etc. >> >> Vl. >> >> >> >> On 27 November 2015 at 11:05, Vlastimil Elias < >> velias at redhat.com> wrote: >> >>> Hi, >>> >>> moving this discussion to the devel forum as it is about the feture >>> development now. >>> >>> Toplevel issue I created for this feature is >>> https://issues.jboss.org/browse/KEYCLOAK-2076 >>> >>> I added some notes and thoughts from my investigation as a comment to >>> the KEYCLOAK-2076, there are some open questions how to implement it. >>> >>> Originally I though I should be able to implement reauth support and >>> provide PR. >>> But I must say I'm not sure now if I'm able to implement it, looks like >>> it is a bit more complicated than I originally expected, so probably some >>> Keycloak core developer should do it. >>> But if you think you will not have resources to do it in 1.8 then I can >>> try it (with your support), as I believe it is a very important feature, >>> and we really want use it. >>> >>> Cheers >>> >>> Vlastimil >>> >>> >>> >>> >>> On 12.11.2015 14:50, Stian Thorgersen wrote: >>> >>> >>> >>> On 12 November 2015 at 14:49, Vlastimil Elias < >>> velias at redhat.com> wrote: >>> >>>> Thanks for quick reply Stian. >>>> >>>> I'm going to create JIRAs for all these things. I can volunter to >>>> implement some parts of this. >>>> >>>> For the last one, it should be probably cool to have "reauthenticate >>>> timeout" setting available in client section for every client (not only >>>> internal admin console and account management). It should allow simple >>>> implementation of "long user sso session" scheme even in environments where >>>> some clients can't be updated to set max_age on protocol level. >>>> >>> >>> Yep, that makes sense >>> >>> >>>> >>>> Vl. >>>> >>>> >>>> On 12.11.2015 14:39, Stian Thorgersen wrote: >>>> >>>> >>>> >>>> On 12 November 2015 at 14:15, Vlastimil Elias < >>>> velias at redhat.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'd like to use long session authentication mechanism known from many >>>>> sites like google. facebook, linked in etc. >>>>> It is about really long user SSO sessions (eg. weeks or even months) >>>>> with reauthentication for important actions when last authentication >>>>> timestamp is older than some limit. >>>>> >>>>> Is this somehow possible with current Keycloak server and Keycloak >>>>> adapters? >>>>> >>>>> I see few subquestions in this problem for our use: >>>>> >>>>> ***** >>>>> open-id connect protocol defines few auth request parameters to support >>>>> this use case, mainly max_age or prompt=login. Are they correctly >>>>> implemented in Keycloak server? >>>>> >>>> >>>> We don't have support for max_age and we only support prompt=none so >>>> these would have to be added >>>> >>>> >>>>> >>>>> >>>>> ***** >>>>> Wildfly/EAP adapter - is it possible and is there some example how to >>>>> use "reauth if auth is older than 30min" action in Java app secured by >>>>> this adapter? Or is info about last auth timestamp somehow available in >>>>> the app? >>>>> >>>> >>>> We don't set auth_time claim ATM so answer is no >>>> >>>> >>>>> >>>>> >>>>> ***** >>>>> Keycloak user account application itself - it is part of the Keycloak >>>>> server, but it contains sensitive actions which typically require >>>>> reathentication in this long session scheme (password change, email >>>>> change, ...). Is it somehow possible to configure Keycloak to force >>>>> timeout reauth for this app? >>>>> >>>> >>>> Not at the moment - but if we add what you want it would also make >>>> sense to add that. Would need to be configurable through the admin console. >>>> Would also be nice to have the same for the admin console itself. >>>> >>>> >>>>> >>>>> Thanks in advance >>>>> >>>>> Vl. >>>>> >>>>> -- >>>>> Vlastimil Elias >>>>> Principal Software Engineer >>>>> Developer Portal Engineering Team >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-user mailing list >>>>> keycloak-user at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-user >>>>> >>>> >>>> >>>> -- >>>> Vlastimil Elias >>>> Principal Software Engineer >>>> Developer Portal Engineering Team >>>> >>>> >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/5ec2fa4e/attachment-0001.html From sthorger at redhat.com Fri Nov 27 09:57:38 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 15:57:38 +0100 Subject: [keycloak-dev] Moved server bits out of integration Message-ID: I've moved everything server related out of integration to a separate wildfly folder. So now integration only contains adapter related modules. Question: should we renamed integration to adapters? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/76533462/attachment.html From bburke at redhat.com Fri Nov 27 10:03:01 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 27 Nov 2015 10:03:01 -0500 Subject: [keycloak-dev] Reset Actions In-Reply-To: References: Message-ID: <565870A5.1050703@redhat.com> Would require the ability to apply a flow to a required action. On 11/27/2015 3:29 AM, Stian Thorgersen wrote: > The new reset actions doesn't require the user to authenticate prior to > performing them. Is it not a bit dangerous that the user can change the > email address without authentication? > > For reset password we obviously need to be able to do it without > requiring authentication, but shouldn't "bypassing" authentication be > limited as much as possible? > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Fri Nov 27 10:07:28 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 27 Nov 2015 10:07:28 -0500 Subject: [keycloak-dev] Implicit and hybrid flow In-Reply-To: References: <56581413.4010709@redhat.com> Message-ID: <565871B0.8070305@redhat.com> On 11/27/2015 3:44 AM, Stian Thorgersen wrote: > > > On 27 November 2015 at 09:28, Marek Posolda > wrote: > > I've sent PR https://github.com/keycloak/keycloak/pull/1885 with initial > support for implicit flow and hybrid flow. > > Some summary: > - Added switches on client in admin console to enable/disable standard > flow, implicit flow, direct grant flow and service accounts. Removed > "direct grants only" switch (Disable both standard and implicit defacto > means enabling of previous "direct grants only") > > > Is direct grant and implicit disabled by default? > > > > - Added more proper support for "response_type" parameter. This is about > _what_ is sent in response from AuthorizationEndpoint to client > application. According to specs, possible values are "code", "id_token", > "token" and some combination of them. See [1] . Until now, we supported > just value "code" (standard AuthorizationCode flow). According to specs, > implicit flow is about "token" and "id_token". Hybrid flow is about code > + some of the tokens. > > > What do we do about backwards compatibility? It's seems we're breaking > the spec currently by including both id_token and refresh_token even > though response_type is just code. > > > In addition to specs, I've added also support for "refresh_token" . So > if you use "response_type=token%20id_token%20refresh_token", the > AuthorizationEndpoint will send all 3 tokens. The OIDC specs doesn't > support sending refreshToken in implicit flow. However in Keycloak case, > we are using access tokens with very short lifespan by default (1 > minute). Having just accessToken and idToken would make the implicit > flow quite unusable, as application will need to re-login each 1 minute. > For example Google doesn't support sending refreshToken in implicit > flow, however it makes much more sense for them as their access token is > valid for 60 minutes. This is even longer than our refresh token (30 > minutes by default). WDYT about support for refresh token? > > > We shouldn't send refresh token in fragment aka support it in implicit. > Maybe we should have different timeouts instead of sending refresh token? > Read this: https://tools.ietf.org/html/rfc6819 And make sure you aren't opening yourself up to any additional attacks. > > > - Added support for "response_mode" parameter. This specifies _how_ are > OIDC parameters (code, state, ... ) sent from AuthorizationEndpoint to > client. More details in specs [2]. Valid values are: > -- query - Params sent in query string ( > http://localhost?code=foo&state=bar ). We always supported this > -- fragment - Params sent in fragment ( > http://localhost#code=foo&state=bar ). This is needed for implicit and > hybrid flow support. Specs doesn't allow to sent tokens in query string. > -- form_post - Params are sent in body of POST method. There is separate > specs for this [3] . I've added just server-side support for this. It > may be interesting to add support on our server adapters, as IMO it's a > bit safer alternative when code+state are sent in POST body instead of > in GET method query string. But I guess this is not big priority? > POST probably works exactly like SAML POST Binding. Its a nice trick. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Fri Nov 27 10:13:56 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 27 Nov 2015 10:13:56 -0500 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: References: <5658210D.1070301@redhat.com> <56583B90.3090904@redhat.com> <56584AED.4040803@redhat.com> Message-ID: <56587334.10209@redhat.com> Why would you need re-auth for REST or even browser clients? Doesn't access token and refresh token timeouts handle this sort of thing? On 11/27/2015 7:25 AM, Stian Thorgersen wrote: > Hmm.. re-auth request for rest endpoints is even more tricky. I think a > rest endpoint would just have to return a 401? Then it's up to the > client to figure out that it needs to re-authenticate. > > On 27 November 2015 at 13:22, Vlastimil Elias > wrote: > > > > On 27.11.2015 12:28, Stian Thorgersen wrote: >> >> >> On 27 November 2015 at 12:16, Vlastimil Elias >> <velias at redhat.com >> > wrote: >> >> Hi, >> >> On 27.11.2015 11:45, Stian Thorgersen wrote: >>> >>> >>> On 27 November 2015 at 10:23, Vlastimil Elias >>> <velias at redhat.com >>> > wrote: >>> >>> Hi, >>> >>> I have two proposals for cleanup of 'Change password' >>> screen in Account >>> app based on my experience with it: >>> >>> 1. remove Cancel button - it has no any meaning on this >>> screen/form, it >>> only reshowns form with empty fields. And also there is a >>> bug, >>> "Password" field is hidden when it is used, which makes >>> whole form unusable. >>> >>> >>> +1 >> >> OK, I'm going to create JIRA and provide PR for this. >> >>> >>> 2. remove validation of current password (remove >>> "Password" field). Two >>> reasons for this: >>> - security impact of this check is small. If attacker >>> is able to >>> compromise Account app then he can always change email >>> and then use >>> "Forgot password" feature to change password >>> - user created over Identity Provider do not know old >>> password >>> (because it is not set) so he is not able to set password >>> using this screen >>> After we implement support for reauthentication >>> (KEYCLOAK-2076) then we >>> should set some reasonable reauth timeout for Account app >>> instead, this >>> will make it more secure at all. >>> >>> >>> -1 Reset password over email may not be enabled at all. We >>> already allow setting password for IdPs login without >>> requiring the existing password. >> Fair enough >>> >>> +1 To suggestion from Thomas - we should ask for password >>> when updating email at least when recover password over email >>> is enabled. >> >> Makes sense, but what to do if user has no password set at >> this point? Don't ask him, or reauthenticate him by other >> available mechanism? >> >> And there are also other "dangerous" operations in Account >> app. Eg. attacker who gains access to it can disable OTP >> without any recheck. This should be protected too. >> I believe whole Account app should be correctly protected by >> reauthentications. Question is if implement it somehow >> specifically in the app, or resolve this generally as part of >> KEYCLOAK-2076. >> >> >> Yep, you're right there's other dangerous operations as well. >> Re-authentication sounds like the correct approach, rather than >> requesting the password again. We could have two configurable >> timeouts in account management console. One that requires re-auth >> for anything (~30 min default), and another that requires re-auth >> for sensitive operations (~5 min). >> >> One thing with re-auth is that if it happens when a user submits a >> form, you then someone have to also remember the values of the >> form. Currently account management is stateless. > > Yep, implementing reauthentication schemes correctly in client apps > may be tricky, mainly for POST requests (and even worse posts with > file upload) and AJAX call handlers etc. > Beside Keycloak Account app we should try to provide reasonable > support in Keycloak adapters, but it may be tricky as related Java > EE specifications (JAAS subsystems, auth related things in Java > Servlet and EJB spec ) do not count with these more complicated > schemes too much. > > And I also thought if reauthentication scheme should be somehow > implemented on REST API levels also. > Eg. some REST endpoint operation will define that it needs reauth > with some timeout. If client app call it with Bearer token then it > should be able to tests last auth timestamp (which should be > possible as it may be in the token as claim) and propagate back info > that it needs reauth (how to map this to HTTP?). Support for this > should be added to the framework like RestEasy then. > > Vl. > >> >> >> Vlastimil >> >>> >>> It seems to be common practice to ask for current password >>> when updating the existing password. >>> >>> >>> If you agree then I can create JIRA issue for this and >>> provide PR. >>> >>> Vlastimil >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >>> >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev at lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >>> >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sthorger at redhat.com Fri Nov 27 10:15:54 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 16:15:54 +0100 Subject: [keycloak-dev] Reset Actions In-Reply-To: <565870A5.1050703@redhat.com> References: <565870A5.1050703@redhat.com> Message-ID: Should we add it though? On 27 November 2015 at 16:03, Bill Burke wrote: > Would require the ability to apply a flow to a required action. > > On 11/27/2015 3:29 AM, Stian Thorgersen wrote: > > The new reset actions doesn't require the user to authenticate prior to > > performing them. Is it not a bit dangerous that the user can change the > > email address without authentication? > > > > For reset password we obviously need to be able to do it without > > requiring authentication, but shouldn't "bypassing" authentication be > > limited as much as possible? > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/2896f47c/attachment.html From sthorger at redhat.com Fri Nov 27 10:19:20 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Fri, 27 Nov 2015 16:19:20 +0100 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: <56587334.10209@redhat.com> References: <5658210D.1070301@redhat.com> <56583B90.3090904@redhat.com> <56584AED.4040803@redhat.com> <56587334.10209@redhat.com> Message-ID: I guess at least the re-auth part is logic that belongs in the client that performs the login. Question though for authentication levels as well as authentication timeout (or whatever you call it) shouldn't a rest service be able to say things like I require the user to have authenticated with password + otp, and to have authenticated within N minutes? On 27 November 2015 at 16:13, Bill Burke wrote: > Why would you need re-auth for REST or even browser clients? Doesn't > access token and refresh token timeouts handle this sort of thing? > > On 11/27/2015 7:25 AM, Stian Thorgersen wrote: > > Hmm.. re-auth request for rest endpoints is even more tricky. I think a > > rest endpoint would just have to return a 401? Then it's up to the > > client to figure out that it needs to re-authenticate. > > > > On 27 November 2015 at 13:22, Vlastimil Elias > > wrote: > > > > > > > > On 27.11.2015 12:28, Stian Thorgersen wrote: > >> > >> > >> On 27 November 2015 at 12:16, Vlastimil Elias > >> <velias at redhat.com > >> > wrote: > >> > >> Hi, > >> > >> On 27.11.2015 11:45, Stian Thorgersen wrote: > >>> > >>> > >>> On 27 November 2015 at 10:23, Vlastimil Elias > >>> <velias at redhat.com > >>> > wrote: > >>> > >>> Hi, > >>> > >>> I have two proposals for cleanup of 'Change password' > >>> screen in Account > >>> app based on my experience with it: > >>> > >>> 1. remove Cancel button - it has no any meaning on this > >>> screen/form, it > >>> only reshowns form with empty fields. And also there is a > >>> bug, > >>> "Password" field is hidden when it is used, which makes > >>> whole form unusable. > >>> > >>> > >>> +1 > >> > >> OK, I'm going to create JIRA and provide PR for this. > >> > >>> > >>> 2. remove validation of current password (remove > >>> "Password" field). Two > >>> reasons for this: > >>> - security impact of this check is small. If attacker > >>> is able to > >>> compromise Account app then he can always change email > >>> and then use > >>> "Forgot password" feature to change password > >>> - user created over Identity Provider do not know old > >>> password > >>> (because it is not set) so he is not able to set password > >>> using this screen > >>> After we implement support for reauthentication > >>> (KEYCLOAK-2076) then we > >>> should set some reasonable reauth timeout for Account app > >>> instead, this > >>> will make it more secure at all. > >>> > >>> > >>> -1 Reset password over email may not be enabled at all. We > >>> already allow setting password for IdPs login without > >>> requiring the existing password. > >> Fair enough > >>> > >>> +1 To suggestion from Thomas - we should ask for password > >>> when updating email at least when recover password over email > >>> is enabled. > >> > >> Makes sense, but what to do if user has no password set at > >> this point? Don't ask him, or reauthenticate him by other > >> available mechanism? > >> > >> And there are also other "dangerous" operations in Account > >> app. Eg. attacker who gains access to it can disable OTP > >> without any recheck. This should be protected too. > >> I believe whole Account app should be correctly protected by > >> reauthentications. Question is if implement it somehow > >> specifically in the app, or resolve this generally as part of > >> KEYCLOAK-2076. > >> > >> > >> Yep, you're right there's other dangerous operations as well. > >> Re-authentication sounds like the correct approach, rather than > >> requesting the password again. We could have two configurable > >> timeouts in account management console. One that requires re-auth > >> for anything (~30 min default), and another that requires re-auth > >> for sensitive operations (~5 min). > >> > >> One thing with re-auth is that if it happens when a user submits a > >> form, you then someone have to also remember the values of the > >> form. Currently account management is stateless. > > > > Yep, implementing reauthentication schemes correctly in client apps > > may be tricky, mainly for POST requests (and even worse posts with > > file upload) and AJAX call handlers etc. > > Beside Keycloak Account app we should try to provide reasonable > > support in Keycloak adapters, but it may be tricky as related Java > > EE specifications (JAAS subsystems, auth related things in Java > > Servlet and EJB spec ) do not count with these more complicated > > schemes too much. > > > > And I also thought if reauthentication scheme should be somehow > > implemented on REST API levels also. > > Eg. some REST endpoint operation will define that it needs reauth > > with some timeout. If client app call it with Bearer token then it > > should be able to tests last auth timestamp (which should be > > possible as it may be in the token as claim) and propagate back info > > that it needs reauth (how to map this to HTTP?). Support for this > > should be added to the framework like RestEasy then. > > > > Vl. > > > >> > >> > >> Vlastimil > >> > >>> > >>> It seems to be common practice to ask for current password > >>> when updating the existing password. > >>> > >>> > >>> If you agree then I can create JIRA issue for this and > >>> provide PR. > >>> > >>> Vlastimil > >>> > >>> -- > >>> Vlastimil Elias > >>> Principal Software Engineer > >>> Developer Portal Engineering Team > >>> > >>> > >>> > >>> _______________________________________________ > >>> keycloak-dev mailing list > >>> keycloak-dev at lists.jboss.org > >>> > >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev > >>> > >>> > >> > >> -- > >> Vlastimil Elias > >> Principal Software Engineer > >> Developer Portal Engineering Team > >> > >> > > > > -- > > Vlastimil Elias > > Principal Software Engineer > > Developer Portal Engineering Team > > > > > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151127/92589e42/attachment-0001.html From bburke at redhat.com Fri Nov 27 10:22:25 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 27 Nov 2015 10:22:25 -0500 Subject: [keycloak-dev] Reset Actions In-Reply-To: References: <565870A5.1050703@redhat.com> Message-ID: <56587531.10605@redhat.com> Its a big change. Would probably have to refactor code a bit too as the current flows assumes authentication then required actions. On 11/27/2015 10:15 AM, Stian Thorgersen wrote: > Should we add it though? > > On 27 November 2015 at 16:03, Bill Burke > wrote: > > Would require the ability to apply a flow to a required action. > > On 11/27/2015 3:29 AM, Stian Thorgersen wrote: > > The new reset actions doesn't require the user to authenticate > prior to > > performing them. Is it not a bit dangerous that the user can > change the > > email address without authentication? > > > > For reset password we obviously need to be able to do it without > > requiring authentication, but shouldn't "bypassing" authentication be > > limited as much as possible? > > > > > > _______________________________________________ > > keycloak-dev mailing list > > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Fri Nov 27 10:23:45 2015 From: bburke at redhat.com (Bill Burke) Date: Fri, 27 Nov 2015 10:23:45 -0500 Subject: [keycloak-dev] Cleanup of 'Change password' screen in Account app In-Reply-To: References: <5658210D.1070301@redhat.com> <56583B90.3090904@redhat.com> <56584AED.4040803@redhat.com> <56587334.10209@redhat.com> Message-ID: <56587581.60209@redhat.com> On 11/27/2015 10:19 AM, Stian Thorgersen wrote: > I guess at least the re-auth part is logic that belongs in the client > that performs the login. > > Question though for authentication levels as well as authentication > timeout (or whatever you call it) shouldn't a rest service be able to > say things like I require the user to have authenticated with password + > otp, and to have authenticated within N minutes? > I think SAML has extensions for that. OIDC doesn't. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bruno at abstractj.org Fri Nov 27 10:45:21 2015 From: bruno at abstractj.org (Bruno Oliveira) Date: Fri, 27 Nov 2015 13:45:21 -0200 Subject: [keycloak-dev] User registration: OTP flow In-Reply-To: References: Message-ID: I think in this case, we can close the issue. OTP form will enable people to use authenticator, but not enforce it during the registration time. On Fri, Nov 27, 2015 at 5:40 AM, Stian Thorgersen wrote: > It's certainly a usability issue then. I thought the opposite that "OTP > Form" was required, but not adding the default action to have it work with > registration as well. > > On 26 November 2015 at 21:24, Bruno Oliveira wrote: >> >> Ahoy, I was looking at this Jira >> https://issues.jboss.org/browse/KEYCLOAK-1998 and trying to reproduce >> the issue reported by Stian. >> >> What I did was: >> >> 1. Get the latest changes from master >> 2. Run mvn clean install -DskipTests=true && mvn -f >> testsuite/integration/pom.xml exec:java -Pkeycloak-server >> 3. Go to "Realm Settings" and enable "User Registration" >> 4. Go to "Authentication" > "Required Actions" >> 5. On Configure Totp mark the checkbox "Default Action" >> 6. Now logout and try to register >> 7. After the registration I get the TOTP screen >> >> Stian was following a different workflow >> >> 1. Get the latest changes from master >> 2. Run mvn clean install -DskipTests=true && mvn -f >> testsuite/integration/pom.xml exec:java -Pkeycloak-server >> 3. Go to "Realm Settings" and enable "User Registration" >> 4. Flows > Browser >> 5. OTP form marked as required >> 6. After the registration I won't get the TOTP screen >> >> After I managed to reproduce the real issue, I got confused about what >> would be the expected behavior in the situation where I have "OTP >> form" as required and "Required actions > Configure Totp > Default >> action" unchecked. >> >> To me it seems like OTP form is unnecessary, but I can be 110% wrong. >> >> Thoughts? >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > From mposolda at redhat.com Sun Nov 29 15:27:53 2015 From: mposolda at redhat.com (Marek Posolda) Date: Sun, 29 Nov 2015 21:27:53 +0100 Subject: [keycloak-dev] Implicit and hybrid flow In-Reply-To: References: <56581413.4010709@redhat.com> <56582813.3010501@redhat.com> Message-ID: <565B5FC9.8080301@redhat.com> On 27/11/15 11:52, Stian Thorgersen wrote: > >> Is direct grant and implicit disabled by default? > Implicit is disabled, but direct grant is enabled by default. This > is just for backwards compatibility, as in 1.6, we have direct > grant defacto enabled for all clients. If we want to have it > disabled by default, we should add big note to migration docs. Or > we can have it enabled for all clients migrated from previous > version, but keep the switch "off" in admin console for new clients? > > > On for old, off for new works for me. Thinking that it's a bit tricky... For example if you import testrealm.json with demo example, the direct grants will be enabled for all clients, but at the same time the switch for newly created clients will be disabled. Looks strange to me. I wonder that for migration, it is more proper to enable direct grants just for the clients, which have "directGrantsOnly" switch enabled? Those are most likely clients, which were in previous version used for direct grants usecase > > > At least, we have people, who wants to login into admin REST API > by default (without need to go to admin-console UI first and > enable direct grant for some client), so I guess this possibility > should be still kept. > > > In reality they should not be using the admin console client to do so. > They should create a separate client for it I think. We need to sort > out some sort of bootstrapping for it though. Or maybe we have a > admin-cli client? +1 for admin-cli client. So how about this: - new clients will have "direct access grant" switch off by default - Clients migrated from previous version will have "direct access grant" just if they had "direct grants only" enabled. So those clients will have "standard=off, direct access grants=on" - New builtin client "admin-cli" will be added to each realm. It will be public client with "standard=off, implicit=off, directAccessGrants=on" and will have same scope like current "security-admin-console" - security-admin-console will have directAccessGrants=off . This will be done automatically during migration from previous version (as it has directGrantsOnly=off in 1.6.1). - Big note will be added to migration guide Marek -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151129/8c4d57c2/attachment.html From sthorger at redhat.com Mon Nov 30 02:28:26 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 30 Nov 2015 08:28:26 +0100 Subject: [keycloak-dev] Implicit and hybrid flow In-Reply-To: <565B5FC9.8080301@redhat.com> References: <56581413.4010709@redhat.com> <56582813.3010501@redhat.com> <565B5FC9.8080301@redhat.com> Message-ID: Sounds good On 29 November 2015 at 21:27, Marek Posolda wrote: > On 27/11/15 11:52, Stian Thorgersen wrote: > > Is direct grant and implicit disabled by default? >> >> Implicit is disabled, but direct grant is enabled by default. This is >> just for backwards compatibility, as in 1.6, we have direct grant defacto >> enabled for all clients. If we want to have it disabled by default, we >> should add big note to migration docs. Or we can have it enabled for all >> clients migrated from previous version, but keep the switch "off" in admin >> console for new clients? >> > > On for old, off for new works for me. > > Thinking that it's a bit tricky... For example if you import > testrealm.json with demo example, the direct grants will be enabled for all > clients, but at the same time the switch for newly created clients will be > disabled. Looks strange to me. > > I wonder that for migration, it is more proper to enable direct grants > just for the clients, which have "directGrantsOnly" switch enabled? Those > are most likely clients, which were in previous version used for direct > grants usecase > > > >> >> At least, we have people, who wants to login into admin REST API by >> default (without need to go to admin-console UI first and enable direct >> grant for some client), so I guess this possibility should be still kept. >> > > In reality they should not be using the admin console client to do so. > They should create a separate client for it I think. We need to sort out > some sort of bootstrapping for it though. Or maybe we have a admin-cli > client? > > +1 for admin-cli client. > > So how about this: > - new clients will have "direct access grant" switch off by default > - Clients migrated from previous version will have "direct access grant" > just if they had "direct grants only" enabled. So those clients will have > "standard=off, direct access grants=on" > - New builtin client "admin-cli" will be added to each realm. It will be > public client with "standard=off, implicit=off, directAccessGrants=on" and > will have same scope like current "security-admin-console" > - security-admin-console will have directAccessGrants=off . This will be > done automatically during migration from previous version (as it has > directGrantsOnly=off in 1.6.1). > - Big note will be added to migration guide > > Marek > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151130/a5159726/attachment.html From sthorger at redhat.com Mon Nov 30 02:29:19 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 30 Nov 2015 08:29:19 +0100 Subject: [keycloak-dev] Reset Actions In-Reply-To: <56587531.10605@redhat.com> References: <565870A5.1050703@redhat.com> <56587531.10605@redhat.com> Message-ID: Ok, let's keep it as is then On 27 November 2015 at 16:22, Bill Burke wrote: > Its a big change. Would probably have to refactor code a bit too as the > current flows assumes authentication then required actions. > > On 11/27/2015 10:15 AM, Stian Thorgersen wrote: > >> Should we add it though? >> >> On 27 November 2015 at 16:03, Bill Burke > > wrote: >> >> Would require the ability to apply a flow to a required action. >> >> On 11/27/2015 3:29 AM, Stian Thorgersen wrote: >> > The new reset actions doesn't require the user to authenticate >> prior to >> > performing them. Is it not a bit dangerous that the user can >> change the >> > email address without authentication? >> > >> > For reset password we obviously need to be able to do it without >> > requiring authentication, but shouldn't "bypassing" authentication >> be >> > limited as much as possible? >> > >> > >> > _______________________________________________ >> > keycloak-dev mailing list >> > keycloak-dev at lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/keycloak-dev >> > >> >> -- >> Bill Burke >> JBoss, a division of Red Hat >> http://bill.burkecentral.com >> _______________________________________________ >> keycloak-dev mailing list >> keycloak-dev at lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >> >> >> > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151130/c2e01c39/attachment-0001.html From sthorger at redhat.com Mon Nov 30 02:32:01 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 30 Nov 2015 08:32:01 +0100 Subject: [keycloak-dev] User registration: OTP flow In-Reply-To: References: Message-ID: My vote is to keep the issue open as it's clearly a usability issue. Neither of us understood how to do it at first. On 27 November 2015 at 16:45, Bruno Oliveira wrote: > I think in this case, we can close the issue. OTP form will enable > people to use authenticator, but not enforce it during the > registration time. > > On Fri, Nov 27, 2015 at 5:40 AM, Stian Thorgersen > wrote: > > It's certainly a usability issue then. I thought the opposite that "OTP > > Form" was required, but not adding the default action to have it work > with > > registration as well. > > > > On 26 November 2015 at 21:24, Bruno Oliveira > wrote: > >> > >> Ahoy, I was looking at this Jira > >> https://issues.jboss.org/browse/KEYCLOAK-1998 and trying to reproduce > >> the issue reported by Stian. > >> > >> What I did was: > >> > >> 1. Get the latest changes from master > >> 2. Run mvn clean install -DskipTests=true && mvn -f > >> testsuite/integration/pom.xml exec:java -Pkeycloak-server > >> 3. Go to "Realm Settings" and enable "User Registration" > >> 4. Go to "Authentication" > "Required Actions" > >> 5. On Configure Totp mark the checkbox "Default Action" > >> 6. Now logout and try to register > >> 7. After the registration I get the TOTP screen > >> > >> Stian was following a different workflow > >> > >> 1. Get the latest changes from master > >> 2. Run mvn clean install -DskipTests=true && mvn -f > >> testsuite/integration/pom.xml exec:java -Pkeycloak-server > >> 3. Go to "Realm Settings" and enable "User Registration" > >> 4. Flows > Browser > >> 5. OTP form marked as required > >> 6. After the registration I won't get the TOTP screen > >> > >> After I managed to reproduce the real issue, I got confused about what > >> would be the expected behavior in the situation where I have "OTP > >> form" as required and "Required actions > Configure Totp > Default > >> action" unchecked. > >> > >> To me it seems like OTP form is unnecessary, but I can be 110% wrong. > >> > >> Thoughts? > >> _______________________________________________ > >> keycloak-dev mailing list > >> keycloak-dev at lists.jboss.org > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151130/9015c761/attachment.html From gerbermichi at me.com Mon Nov 30 02:41:13 2015 From: gerbermichi at me.com (Michael Gerber) Date: Mon, 30 Nov 2015 07:41:13 +0000 (GMT) Subject: [keycloak-dev] class not found Message-ID: <305bd269-08b0-4549-a5ad-15188867cf85@me.com> Hi all, I had to add the javax.mail.api module to the keycloak-services today, otherwise the reset password mail could not be sent. (Because the class javax.mail.Multipart?could?not be found) I am using wildlfy 10 with the newest keycloak version.? Is this a known issue? Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151130/7b0ddae6/attachment.html From sthorger at redhat.com Mon Nov 30 03:03:37 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 30 Nov 2015 09:03:37 +0100 Subject: [keycloak-dev] class not found In-Reply-To: <305bd269-08b0-4549-a5ad-15188867cf85@me.com> References: <305bd269-08b0-4549-a5ad-15188867cf85@me.com> Message-ID: We do not support deploying Keycloak server on WildFly 10 yet. On 30 November 2015 at 08:41, Michael Gerber wrote: > Hi all, > > I had to add the javax.mail.api module to the keycloak-services today, > otherwise the reset password mail could not be sent. (Because the class > javax.mail.Multipart could not be found) > > I am using wildlfy 10 with the newest keycloak version. > > Is this a known issue? > > Michael > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151130/3bb3f158/attachment.html From gerbermichi at me.com Mon Nov 30 03:20:00 2015 From: gerbermichi at me.com (Michael Gerber) Date: Mon, 30 Nov 2015 08:20:00 +0000 (GMT) Subject: [keycloak-dev] class not found Message-ID: <724eee1c-d567-4ce5-86b5-0c88cee34032@me.com> Ok thank you. But it worked a few weeks ago. I think?it is since the "KEYCLOAK-1758 add-user script" commit. Am 30. November 2015 um 09:03 schrieb Stian Thorgersen : We do not support deploying Keycloak server on WildFly 10 yet. On 30 November 2015 at 08:41, Michael Gerber wrote: Hi all, I had to add the javax.mail.api module to the keycloak-services today, otherwise the reset password mail could not be sent. (Because the class javax.mail.Multipart?could?not be found) I am using wildlfy 10 with the newest keycloak version.? Is this a known issue? Michael _______________________________________________ keycloak-dev mailing list keycloak-dev at lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151130/f3a6cd13/attachment.html From sthorger at redhat.com Mon Nov 30 06:45:04 2015 From: sthorger at redhat.com (Stian Thorgersen) Date: Mon, 30 Nov 2015 12:45:04 +0100 Subject: [keycloak-dev] Disabled admin console tests on Travis Message-ID: Admin console tests are to unstable at the moment so I've disabled them on Travis. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151130/02b551ee/attachment.html From ayoung at redhat.com Mon Nov 30 17:07:35 2015 From: ayoung at redhat.com (Adam Young) Date: Mon, 30 Nov 2015 17:07:35 -0500 Subject: [keycloak-dev] Keycloak in Tomcat server In-Reply-To: References: Message-ID: <565CC8A7.6050109@redhat.com> On 11/26/2015 03:05 AM, Stian Thorgersen wrote: > In enterprise they should recommend using a standalone SSO server. > It's more secure and scalable. You wouldn't embed your database into > Tomcat would you? It's also more reliable and less error prone as you > are using the exact bits that we test. > > It has been achieved in the past, but initially we relied less on > features from WildFly. Keycloak server is no longer a WAR that's > deployed onto the WildFly app server, instead we rely on the core bits > of WildFly to create our own dedicated server. > > In theory it would be possible to get it to work on top of Tomcat with > some considerable amount of effort. However, as we add new features in > the future that rely on features in WildFly you would have new issues. > Further we would not be able to help you if you have any issues. Can you document this? I assume the JMX components are the biggest aspect, but what else requires a full JEE app server? Is there any JMS integration? > > On 26 November 2015 at 08:54, mohammed althaf > wrote: > > Hi Stian, > > Thanks for your quick reply.Yeah i have tried the Tomcat adapters > for keycloak Integration. > > I just want to explore the option to deploy it in tomcat itself > ,since in enterprise if they already use tomcat for applications > they wont recommend to use another server for SSO alone. > > From your knowledge has anyone achieved that ?? > > Thanks, > Althaf > > On 26 November 2015 at 15:40, Stian Thorgersen > > wrote: > > Hi, > > You do not need to deploy the server to Tomcat to be able to > use Keycloak in your apps deployed to Tomcat. > > We only support deploying Keycloak server to latest version of > WildFly and JBoss EAP. Further we highly recommend using the > standalone download of Keycloak and running it separately to > your applications. > > On 26 November 2015 at 07:24, mohammed althaf > > wrote: > > Hi Team, > > I was able to use keycloak for sso seamlessly in jboss > /wildfly server. > > Since my web application is running in tomcat ,i am trying > to deploy the keycloak server as well in Tomcat, > > I followed > https://reachmnadeem.wordpress.com/2015/01/14/deploying-keycloak-in-tomcat/ > > and i was able to spin up the keycloak in tomcat > server.But it was using the 1.0.2 version. > > So i am trying to deploy the keycloak server 1.6.1.I faced > the below issues. > > 1.Failed to provide mem for realmcache(please check failed > to find provider log) > > So i removed the cache from keycloak-server.json > Not sure it is fine?? > > 2 .Then i was able to procced but it again failed with > h2.db exception(please check lockout.log) > > It will be great if you guys can help me resolve this issue. > > Thanks, > Althaf > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/keycloak-dev > > > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151130/6acf08eb/attachment-0001.html From bburke at redhat.com Mon Nov 30 17:24:09 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 30 Nov 2015 17:24:09 -0500 Subject: [keycloak-dev] Keycloak in Tomcat server In-Reply-To: <565CC8A7.6050109@redhat.com> References: <565CC8A7.6050109@redhat.com> Message-ID: <565CCC89.60009@redhat.com> On 11/30/2015 5:07 PM, Adam Young wrote: > On 11/26/2015 03:05 AM, Stian Thorgersen wrote: >> In enterprise they should recommend using a standalone SSO server. >> It's more secure and scalable. You wouldn't embed your database into >> Tomcat would you? It's also more reliable and less error prone as you >> are using the exact bits that we test. >> >> It has been achieved in the past, but initially we relied less on >> features from WildFly. Keycloak server is no longer a WAR that's >> deployed onto the WildFly app server, instead we rely on the core bits >> of WildFly to create our own dedicated server. >> >> In theory it would be possible to get it to work on top of Tomcat with >> some considerable amount of effort. However, as we add new features in >> the future that rely on features in WildFly you would have new issues. >> Further we would not be able to help you if you have any issues. > > Can you document this? I assume the JMX components are the biggest > aspect, but what else requires a full JEE app server? Is there any JMS > integration? > JMX is not involved. Infinispan (caching), JPA, datasources, servlet, JAX-RS. Wildfly/JBoss is also set to run out of the box in a cluster and managable in a domain ( a cluster) out of the box. Not to mention all the classloader isolation you DO NOT get with Tomcat. Finally all the built in patch management that comes with Wildfly/JBoss. Then there's developers that will want to deploy integration/extension plugins. We can also leverage Wildfly's deployment engine for that too. Running Keycloak Auth Server in Tomcat/Jetty would actually not be a very smart thing to do. There are huge advantages to running within Wildfly/JBoss. The only disadvantage is the size of the distro. There is no performance penalty. We have looked into trimming the Wildfly distro, but nixed that because it puts a huge burden on productization. Its just much easier for them if we just layer on top of the full app server. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From ayoung at redhat.com Mon Nov 30 19:33:40 2015 From: ayoung at redhat.com (Adam Young) Date: Mon, 30 Nov 2015 19:33:40 -0500 Subject: [keycloak-dev] [keycloak-user] How to implement long user sso sessions with reauthentication for important actions? In-Reply-To: References: <5644910D.9020100@redhat.com> <564498DE.6010805@redhat.com> <56582AEE.4020508@redhat.com> <56584E5C.1060802@redhat.com> <565851D0.40008@redhat.com> Message-ID: <565CEAE4.3040003@redhat.com> On 11/27/2015 07:57 AM, Stian Thorgersen wrote: > Current flow checks if cookie is there and if it is it's all happy. > We'd need a different flow that can require the re-auth to take place > even if cookie is there. Also, with re-auth flow user should be able > to cancel or just navigate away and continue using the session as if > nothing has happend (with the other apps that's happy with the old > session that is) > > On 27 November 2015 at 13:51, Vlastimil Elias > wrote: > > > > On 27.11.2015 13:46, Stian Thorgersen wrote: >> >> >> On 27 November 2015 at 13:36, Vlastimil Elias > > wrote: >> >> >> >> On 27.11.2015 12:22, Stian Thorgersen wrote: >>> Initially when you proposed this feature I underestimated >>> how complicated it would be as well. Reading the discussion >>> on the issue you created I've got a few comments: >>> >>> * We need a new flow - users could want to require re-auth >>> with password, or with totp, or with both, etc.. >> Agree, but at least first step must be same with Browser flow >> - check of cookie, and then redirect to another flow if >> necessary. Not sure if current flow engine supports this. >> >> >> Current browser flow would just automatically log-in as the >> cookie is there > > yep, but new Reauthentication flow must also check if user is > logged in currently (over cookie). If not logged in then it should > go to current Browser flow to login him. If logged in then it must > continue in Reathentication flow. So check "is user currently > logged in?" should be same for both flows. > > Vl > If not logged in, it is not reauthentication, it is delegation. If I ask for a long running task, and I am not around when something needs to be done on my behalf, I need to make sure I've delegated to the remote principal the ability to perform this task, and that remote principal needs to be able to confirm it. > > >> >> >>> * There's probably quite a few corner cases to cover - >>> you've already mentioned a few >>> >>> It's a very nice to have feature, but I doubt have time to >>> work on it for 1.8. >>> >>> Is it something we can post-pone to 2.x or do you really >>> need it now? >> >> We do not need it just now, but should be cool to have it in >> some reasonable time (half of year?). >> Not sure when you plan to start work on 2.x, probably >> sometime around half of next year? >> >> As it is really complex then we probably should break it into >> smaller chunks and implement it per partes over the releases. >> I can imagine first version with basic flow support with Form >> password auth only, then add other parts like support for >> Identity Brokers, OTP, Kerberos, required actions etc. >> >> Vl. >> >> >>> >>> On 27 November 2015 at 11:05, Vlastimil Elias >>> > wrote: >>> >>> Hi, >>> >>> moving this discussion to the devel forum as it is about >>> the feture development now. >>> >>> Toplevel issue I created for this feature is >>> https://issues.jboss.org/browse/KEYCLOAK-2076 >>> >>> I added some notes and thoughts from my investigation as >>> a comment to the KEYCLOAK-2076, there are some open >>> questions how to implement it. >>> >>> Originally I though I should be able to implement reauth >>> support and provide PR. >>> But I must say I'm not sure now if I'm able to implement >>> it, looks like it is a bit more complicated than I >>> originally expected, so probably some Keycloak core >>> developer should do it. >>> But if you think you will not have resources to do it in >>> 1.8 then I can try it (with your support), as I believe >>> it is a very important feature, and we really want use it. >>> >>> Cheers >>> >>> Vlastimil >>> >>> >>> >>> >>> On 12.11.2015 14:50, Stian Thorgersen wrote: >>>> >>>> >>>> On 12 November 2015 at 14:49, Vlastimil Elias >>>> > wrote: >>>> >>>> Thanks for quick reply Stian. >>>> >>>> I'm going to create JIRAs for all these things. I >>>> can volunter to implement some parts of this. >>>> >>>> For the last one, it should be probably cool to >>>> have "reauthenticate timeout" setting available in >>>> client section for every client (not only internal >>>> admin console and account management). It should >>>> allow simple implementation of "long user sso >>>> session" scheme even in environments where some >>>> clients can't be updated to set max_age on protocol >>>> level. >>>> >>>> >>>> Yep, that makes sense >>>> >>>> >>>> Vl. >>>> >>>> >>>> On 12.11.2015 14:39, Stian Thorgersen wrote: >>>>> >>>>> >>>>> On 12 November 2015 at 14:15, Vlastimil Elias >>>>> > wrote: >>>>> >>>>> Hi, >>>>> >>>>> I'd like to use long session authentication >>>>> mechanism known from many >>>>> sites like google. facebook, linked in etc. >>>>> It is about really long user SSO sessions (eg. >>>>> weeks or even months) >>>>> with reauthentication for important actions >>>>> when last authentication >>>>> timestamp is older than some limit. >>>>> >>>>> Is this somehow possible with current Keycloak >>>>> server and Keycloak adapters? >>>>> >>>>> I see few subquestions in this problem for our >>>>> use: >>>>> >>>>> ***** >>>>> open-id connect protocol defines few auth >>>>> request parameters to support >>>>> this use case, mainly max_age or prompt=login. >>>>> Are they correctly >>>>> implemented in Keycloak server? >>>>> >>>>> >>>>> We don't have support for max_age and we only >>>>> support prompt=none so these would have to be added >>>>> >>>>> >>>>> >>>>> ***** >>>>> Wildfly/EAP adapter - is it possible and is >>>>> there some example how to >>>>> use "reauth if auth is older than 30min" >>>>> action in Java app secured by >>>>> this adapter? Or is info about last auth >>>>> timestamp somehow available in >>>>> the app? >>>>> >>>>> >>>>> We don't set auth_time claim ATM so answer is no >>>>> >>>>> >>>>> >>>>> ***** >>>>> Keycloak user account application itself - it >>>>> is part of the Keycloak >>>>> server, but it contains sensitive actions >>>>> which typically require >>>>> reathentication in this long session scheme >>>>> (password change, email >>>>> change, ...). Is it somehow possible to >>>>> configure Keycloak to force >>>>> timeout reauth for this app? >>>>> >>>>> >>>>> Not at the moment - but if we add what you want it >>>>> would also make sense to add that. Would need to >>>>> be configurable through the admin console. Would >>>>> also be nice to have the same for the admin >>>>> console itself. >>>>> >>>>> >>>>> Thanks in advance >>>>> >>>>> Vl. >>>>> >>>>> -- >>>>> Vlastimil Elias >>>>> Principal Software Engineer >>>>> Developer Portal Engineering Team >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> keycloak-user mailing list >>>>> keycloak-user at lists.jboss.org >>>>> >>>>> https://lists.jboss.org/mailman/listinfo/keycloak-user >>>>> >>>>> >>>> >>>> -- >>>> Vlastimil Elias >>>> Principal Software Engineer >>>> Developer Portal Engineering Team >>>> >>>> >>> >>> -- >>> Vlastimil Elias >>> Principal Software Engineer >>> Developer Portal Engineering Team >>> >>> >> >> -- >> Vlastimil Elias >> Principal Software Engineer >> Developer Portal Engineering Team >> >> > > -- > Vlastimil Elias > Principal Software Engineer > Developer Portal Engineering Team > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev at lists.jboss.org > https://lists.jboss.org/mailman/listinfo/keycloak-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151130/4b1f562e/attachment-0001.html From ayoung at redhat.com Mon Nov 30 20:56:23 2015 From: ayoung at redhat.com (Adam Young) Date: Mon, 30 Nov 2015 20:56:23 -0500 Subject: [keycloak-dev] Keycloak in Tomcat server In-Reply-To: <565CCC89.60009@redhat.com> References: <565CC8A7.6050109@redhat.com> <565CCC89.60009@redhat.com> Message-ID: <565CFE47.1060603@redhat.com> On 11/30/2015 05:24 PM, Bill Burke wrote: > Infinispan (caching), JPA, datasources, servlet, JAX-RS. Wildfly/JBoss > is also set to run out of the box in a cluster and managable in a domain > ( a cluster) out of the box. Not to mention all the classloader > isolation you DO NOT get with Tomcat. Finally all the built in patch > management that comes with Wildfly/JBoss. Then there's developers that > will want to deploy integration/extension plugins. We can also leverage > Wildfly's deployment engine for that too. > > Running Keycloak Auth Server in Tomcat/Jetty would actually not be a > very smart thing to do. There are huge advantages to running within > Wildfly/JBoss. The only disadvantage is the size of the distro. There > is no performance penalty. In order to deploy Keycloak as a partner to FreeIPA, it needs to be managed in the same manner as FreeIPA. They are two different deployment strategies, with different management tooling around each. Dogtag is an example of Tomcat only based deployment that is managed via RPMs, with a specially hardened Tomcat container that is necessary to pass Common Criteria and FIPS 140 certifications; making those changes to JBoss would be awesome, but perhaps far more of an engineering effort than any of us care to make. I am personally a fan of JBoss based deployments, but a Tomcat only is more practical from a Fedora and CentOS starting point. We see this same issue come up with all of the language specific package and patch managers. We can't deploy Python code from PIP, Ruby via Gems, or Perl from CPAN; they all get packaged first. The extra work ensures that nothing binary-only sneaks in, that all licenses get reviewed, and that someone from outside the team reviews the packaging to ensure it meets distribution standards. > > We have looked into trimming the Wildfly distro, but nixed that because > it puts a huge burden on productization. Its just much easier for them > if we just layer on top of the full app server. From bburke at redhat.com Mon Nov 30 21:45:58 2015 From: bburke at redhat.com (Bill Burke) Date: Mon, 30 Nov 2015 21:45:58 -0500 Subject: [keycloak-dev] Keycloak in Tomcat server In-Reply-To: <565CFE47.1060603@redhat.com> References: <565CC8A7.6050109@redhat.com> <565CCC89.60009@redhat.com> <565CFE47.1060603@redhat.com> Message-ID: <565D09E6.8@redhat.com> On 11/30/2015 8:56 PM, Adam Young wrote: > On 11/30/2015 05:24 PM, Bill Burke wrote: >> Infinispan (caching), JPA, datasources, servlet, JAX-RS. Wildfly/JBoss >> is also set to run out of the box in a cluster and managable in a domain >> ( a cluster) out of the box. Not to mention all the classloader >> isolation you DO NOT get with Tomcat. Finally all the built in patch >> management that comes with Wildfly/JBoss. Then there's developers that >> will want to deploy integration/extension plugins. We can also leverage >> Wildfly's deployment engine for that too. >> >> Running Keycloak Auth Server in Tomcat/Jetty would actually not be a >> very smart thing to do. There are huge advantages to running within >> Wildfly/JBoss. The only disadvantage is the size of the distro. There >> is no performance penalty. > > > In order to deploy Keycloak as a partner to FreeIPA, it needs to be > managed in the same manner as FreeIPA. > While we do have somebody working on FreeIPA integration, we won't be rearchitecting our whole deployment strategy just to "partner" with Free IPA. Especially considering we've had ZERO inquiries for FreeIPA integration. > They are two different deployment strategies, with different management > tooling around each. Dogtag is an example of Tomcat only based > deployment that is managed via RPMs, with a specially hardened Tomcat > container that is necessary to pass Common Criteria and FIPS 140 > certifications; making those changes to JBoss would be awesome, but > perhaps far more of an engineering effort than any of us care to make. > I'm not sure you are correct of your assessment of JBoss. It has been in the product line for 9+ years now. I think it may have already passed Common Criteria years ago. I know it was something they were aiming for back in 2007-2008. To put it bluntly, getting Keycloak running on Tomcat is not an engineering effort that any of us on Keycloak care to make :) I see ZERO technical advantages and it is just not how we want our userbase consuming Keycloak. We will not be supporting it in product. Feel free to take it up with our PM if you feel so strongly about it. But I feel equally as strong that deploying on Tomcat is a bad idea. > I am personally a fan of JBoss based deployments, but a Tomcat only is > more practical from a Fedora and CentOS starting point. > I honestly do not care how practical Tomcat-only is for Fedora or CentOS. It is more important to solve customer's technical needs than the needs of the fedora community. We *have* managed to build a very robust community without being distributed by Fedora. About 3000-4000 downloads per month with a strong backlog of customers asking for support. A large amount of these users don't even run on Linux. > We see this same issue come up with all of the language specific package > and patch managers. We can't deploy Python code from PIP, Ruby via > Gems, or Perl from CPAN; they all get packaged first. The extra work > ensures that nothing binary-only sneaks in, that all licenses get > reviewed, and that someone from outside the team reviews the packaging > to ensure it meets distribution standards. > You do know that JBoss has been in the product line for 9+ years now? And Keycloak will be piggybacking off of the JBoss productization process? -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com