3 a.m.
After your last email with regards to removing the import button from
client create page I had an idea.
How about we do the following:
Import/export single
--------------------------
On realm, client, identity provider and user federation create pages we add
the import button. This will prefill the form and let the user review
before importing. This is how realm and client works now. We'd also add a
link to export a single entity when displaying it in the admin console
(next to the delete icon).
Batch export
-----------------
When exporting a realm you can select what you want to export. The option
would include realm settings, clients, identity brokers, user federation,
users, credentials. Further there would be an option if export would be
done to a file or a json download. If export to file is selected you would
get the option to export credentials for users, if json download is
selected that option would be disabled.
Batch import
-----------------
We should have options to import a realm as well as import into an existing
realm. For this we should have an option to select what happens if
resources exists (for example client with client-id exists, or user with
username exists). Options could be replace, skip, warn, error, etc..
Finally I was also thinking about an option where we'd have a import
directory on the server. Any files in this would be imported on startup.
Once imported we'd add a "<filename>.imported" or
"<filename>.failed". Same
here it would be nice to be able to somehow specify the strategy if the
resource exists.
3:06 a.m.
I think all of these sound useful!
May I suggest another useful option when importing realm or client, which
is to re-generate keys and secrets?
Best regards,
Thomas
On Wed, Oct 21, 2015 at 11:00 AM, Stian Thorgersen <sthorger(a)redhat.com>
wrote:
After your last email with regards to removing the import button
from
client create page I had an idea.
How about we do the following:
Import/export single
--------------------------
On realm, client, identity provider and user federation create pages we
add the import button. This will prefill the form and let the user review
before importing. This is how realm and client works now. We'd also add a
link to export a single entity when displaying it in the admin console
(next to the delete icon).
Batch export
-----------------
When exporting a realm you can select what you want to export. The option
would include realm settings, clients, identity brokers, user federation,
users, credentials. Further there would be an option if export would be
done to a file or a json download. If export to file is selected you would
get the option to export credentials for users, if json download is
selected that option would be disabled.
Batch import
-----------------
We should have options to import a realm as well as import into an
existing realm. For this we should have an option to select what happens if
resources exists (for example client with client-id exists, or user with
username exists). Options could be replace, skip, warn, error, etc..
Finally I was also thinking about an option where we'd have a import
directory on the server. Any files in this would be imported on startup.
Once imported we'd add a "<filename>.imported" or
"<filename>.failed". Same
here it would be nice to be able to somehow specify the strategy if the
resource exists.
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
5 a.m.
Those are nice additional options we could have. Simply have a checkbox to
re-generate realm keys and another checkbox to re-generate client secrets
(if a client is using jwt auth then we shouldn't re-generate the keys for
the client as we don't store the private key).
On 21 October 2015 at 10:06, Thomas Raehalme <
thomas.raehalme(a)aitiofinland.com> wrote:
I think all of these sound useful!
May I suggest another useful option when importing realm or client, which
is to re-generate keys and secrets?
Best regards,
Thomas
On Wed, Oct 21, 2015 at 11:00 AM, Stian Thorgersen <sthorger(a)redhat.com>
wrote:
> After your last email with regards to removing the import button from
> client create page I had an idea.
>
> How about we do the following:
>
>
> Import/export single
> --------------------------
> On realm, client, identity provider and user federation create pages we
> add the import button. This will prefill the form and let the user review
> before importing. This is how realm and client works now. We'd also add a
> link to export a single entity when displaying it in the admin console
> (next to the delete icon).
>
> Batch export
> -----------------
> When exporting a realm you can select what you want to export. The option
> would include realm settings, clients, identity brokers, user federation,
> users, credentials. Further there would be an option if export would be
> done to a file or a json download. If export to file is selected you would
> get the option to export credentials for users, if json download is
> selected that option would be disabled.
>
> Batch import
> -----------------
> We should have options to import a realm as well as import into an
> existing realm. For this we should have an option to select what happens if
> resources exists (for example client with client-id exists, or user with
> username exists). Options could be replace, skip, warn, error, etc..
>
>
> Finally I was also thinking about an option where we'd have a import
> directory on the server. Any files in this would be imported on startup.
> Once imported we'd add a "<filename>.imported" or
"<filename>.failed". Same
> here it would be nice to be able to somehow specify the strategy if the
> resource exists.
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
7:15 a.m.
I like those ideas too. Some have already been talked about but
regarded as "nice to have".
The question is do we want me to spend extra weeks on all those features
or do we want to get started on CLI?
Right now, I have batch import implemented for Users, Clients, and
Identity Providers. It's easy to add the replace, skip, error feature,
so I'll probably spend a couple of extra hours today doing that.
Personally, I think the best approach is to implement the simplest
possible version of the feature and then get feedback to see what
enhancements are really needed. If you want to try out the import
feature, It's here:
https://github.com/ssilvert/keycloak/tree/user-import-export
On 10/21/2015 6:00 AM, Stian Thorgersen wrote:
Those are nice additional options we could have. Simply have a
checkbox to re-generate realm keys and another checkbox to re-generate
client secrets (if a client is using jwt auth then we shouldn't
re-generate the keys for the client as we don't store the private key).
On 21 October 2015 at 10:06, Thomas Raehalme
<thomas.raehalme(a)aitiofinland.com
<mailto:thomas.raehalme@aitiofinland.com>> wrote:
I think all of these sound useful!
May I suggest another useful option when importing realm or
client, which is to re-generate keys and secrets?
Best regards,
Thomas
On Wed, Oct 21, 2015 at 11:00 AM, Stian Thorgersen
<sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote:
After your last email with regards to removing the import
button from client create page I had an idea.
How about we do the following:
Import/export single
--------------------------
On realm, client, identity provider and user federation create
pages we add the import button. This will prefill the form and
let the user review before importing. This is how realm and
client works now. We'd also add a link to export a single
entity when displaying it in the admin console (next to the
delete icon).
Batch export
-----------------
When exporting a realm you can select what you want to export.
The option would include realm settings, clients, identity
brokers, user federation, users, credentials. Further there
would be an option if export would be done to a file or a json
download. If export to file is selected you would get the
option to export credentials for users, if json download is
selected that option would be disabled.
Batch import
-----------------
We should have options to import a realm as well as import
into an existing realm. For this we should have an option to
select what happens if resources exists (for example client
with client-id exists, or user with username exists). Options
could be replace, skip, warn, error, etc..
Finally I was also thinking about an option where we'd have a
import directory on the server. Any files in this would be
imported on startup. Once imported we'd add a
"<filename>.imported" or "<filename>.failed".
Same here it
would be nice to be able to somehow specify the strategy if
the resource exists.
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
7:27 a.m.
I'd like to get import/export done properly. The addition of being able to
add bits and pieces to import in a directory would be really helpful on
Docker/OpenShift/etc..
Besides, CLI is on hold until we decide what to do. We're not going to
decide on that in the next week weeks. I'm not sure we have the resources
available to do CLI properly before Christmas, so it would probably be
better to wait.
On 21 October 2015 at 14:15, Stan Silvert <ssilvert(a)redhat.com> wrote:
I like those ideas too. Some have already been talked about but
regarded
as "nice to have".
The question is do we want me to spend extra weeks on all those features
or do we want to get started on CLI?
Right now, I have batch import implemented for Users, Clients, and
Identity Providers. It's easy to add the replace, skip, error feature, so
I'll probably spend a couple of extra hours today doing that.
Personally, I think the best approach is to implement the simplest
possible version of the feature and then get feedback to see what
enhancements are really needed. If you want to try out the import feature,
It's here:
https://github.com/ssilvert/keycloak/tree/user-import-export
On 10/21/2015 6:00 AM, Stian Thorgersen wrote:
Those are nice additional options we could have. Simply have a checkbox to
re-generate realm keys and another checkbox to re-generate client secrets
(if a client is using jwt auth then we shouldn't re-generate the keys for
the client as we don't store the private key).
On 21 October 2015 at 10:06, Thomas Raehalme <
thomas.raehalme(a)aitiofinland.com> wrote:
> I think all of these sound useful!
>
> May I suggest another useful option when importing realm or client, which
> is to re-generate keys and secrets?
>
> Best regards,
> Thomas
>
> On Wed, Oct 21, 2015 at 11:00 AM, Stian Thorgersen <sthorger(a)redhat.com>
> wrote:
>
>> After your last email with regards to removing the import button from
>> client create page I had an idea.
>>
>> How about we do the following:
>>
>>
>> Import/export single
>> --------------------------
>> On realm, client, identity provider and user federation create pages we
>> add the import button. This will prefill the form and let the user review
>> before importing. This is how realm and client works now. We'd also add a
>> link to export a single entity when displaying it in the admin console
>> (next to the delete icon).
>>
>> Batch export
>> -----------------
>> When exporting a realm you can select what you want to export. The
>> option would include realm settings, clients, identity brokers, user
>> federation, users, credentials. Further there would be an option if export
>> would be done to a file or a json download. If export to file is selected
>> you would get the option to export credentials for users, if json download
>> is selected that option would be disabled.
>>
>> Batch import
>> -----------------
>> We should have options to import a realm as well as import into an
>> existing realm. For this we should have an option to select what happens if
>> resources exists (for example client with client-id exists, or user with
>> username exists). Options could be replace, skip, warn, error, etc..
>>
>>
>> Finally I was also thinking about an option where we'd have a import
>> directory on the server. Any files in this would be imported on startup.
>> Once imported we'd add a "<filename>.imported" or
"<filename>.failed". Same
>> here it would be nice to be able to somehow specify the strategy if the
>> resource exists.
>>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>
>
7:29 a.m.
On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen <sthorger(a)redhat.com>
wrote:
I'd like to get import/export done properly. The addition of
being able to
add bits and pieces to import in a directory would be really helpful on
Docker/OpenShift/etc..
I had similar things in mind when I suggested the re-generation of keys and
secrets. You could define a template which you'd use in a process for new
deployments.
Best regards,
Thomas
7:33 a.m.
Can you elaborate a bit on that?
On 21 October 2015 at 14:29, Thomas Raehalme <
thomas.raehalme(a)aitiofinland.com> wrote:
On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen <sthorger(a)redhat.com>
wrote:
> I'd like to get import/export done properly. The addition of being able
> to add bits and pieces to import in a directory would be really helpful on
> Docker/OpenShift/etc..
>
I had similar things in mind when I suggested the re-generation of keys
and secrets. You could define a template which you'd use in a process for
new deployments.
Best regards,
Thomas
7:40 a.m.
If you deploy the same application multiple times for different customers
you could have a configuration template containing all the common bits and
pieces, but have Keycloak generate keys and secrets when you import the
configuration.
Best regards,
Thomas
On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen <sthorger(a)redhat.com>
wrote:
Can you elaborate a bit on that?
On 21 October 2015 at 14:29, Thomas Raehalme <
thomas.raehalme(a)aitiofinland.com> wrote:
>
>
> On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen <sthorger(a)redhat.com>
> wrote:
>
>> I'd like to get import/export done properly. The addition of being able
>> to add bits and pieces to import in a directory would be really helpful on
>> Docker/OpenShift/etc..
>>
>
> I had similar things in mind when I suggested the re-generation of keys
> and secrets. You could define a template which you'd use in a process for
> new deployments.
>
> Best regards,
> Thomas
>
--
*Thomas Raehalme*
*CTO, teknologiajohtaja*
Mobile +358 40 545 0605
*Aitio Finland Oy*
Väinönkatu 26 A
40100 JYVÄSKYLÄ, Finland
Tel. +358 10 322 0040
www.aitiofinland.com <http://www.codecenter.fi>
*Codecenter on nyt Aitio – me kun ei vain koodata!*
7:44 a.m.
I've just added client registration services in 1.6 which should be more
useful for that. There's also a Java library. Basically admins and service
accounts with the role create-client can create new clients, while clients
themselves have permissions to update their config.
You could have a client template that you change the client-id only then
use this endpoint to register the client.
On 21 October 2015 at 14:40, Thomas Raehalme <
thomas.raehalme(a)aitiofinland.com> wrote:
If you deploy the same application multiple times for different
customers
you could have a configuration template containing all the common bits and
pieces, but have Keycloak generate keys and secrets when you import the
configuration.
Best regards,
Thomas
On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen <sthorger(a)redhat.com>
wrote:
> Can you elaborate a bit on that?
>
> On 21 October 2015 at 14:29, Thomas Raehalme <
> thomas.raehalme(a)aitiofinland.com> wrote:
>
>>
>>
>> On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen <sthorger(a)redhat.com>
>> wrote:
>>
>>> I'd like to get import/export done properly. The addition of being able
>>> to add bits and pieces to import in a directory would be really helpful on
>>> Docker/OpenShift/etc..
>>>
>>
>> I had similar things in mind when I suggested the re-generation of keys
>> and secrets. You could define a template which you'd use in a process for
>> new deployments.
>>
>> Best regards,
>> Thomas
>>
>
>
--
*Thomas Raehalme*
*CTO, teknologiajohtaja*
Mobile +358 40 545 0605
*Aitio Finland Oy*
Väinönkatu 26 A
40100 JYVÄSKYLÄ, Finland
Tel. +358 10 322 0040
www.aitiofinland.com <http://www.codecenter.fi>
*Codecenter on nyt Aitio – me kun ei vain koodata!*
7:22 a.m.
Sorry for late response. Is the scope of this task also to improve big
long-running export/import ? We already have someone with 75.000 users
in DB.
It seems that export/import is kind of task, which can be easily
"paginated" (Export of user1 - user500 is page1, export of
user501-user1000 is page2 etc). We may have the table where you can see
the progress of long-running export/import job and how many users
(pages) are already exported.
Besides that we may support:
- Concurrency (more worker threads. Each worker doing an export of
different page)
- Cluster scalability (Each cluster node takes some pages and helps with
the overall export task)
- Cluster failover (When some cluster node crashes during export job,
the whole job is not canceled but continue on other cluster nodes
without problem)
I've already addressed the concurrency, scalability and failover with
the InfinispanUserSessionInitializer, which is used at startup for
pre-load offline sessions from DB. It's based on infinispan Distributed
Executor service. I think that with some minor changes (maybe even
without them) it can be reused for any long running "paginatable" job
(export/import, sync of big number of users from federationProvider etc)
Marek
On 21/10/15 14:44, Stian Thorgersen wrote:
I've just added client registration services in 1.6 which should
be
more useful for that. There's also a Java library. Basically admins
and service accounts with the role create-client can create new
clients, while clients themselves have permissions to update their
config.
You could have a client template that you change the client-id only
then use this endpoint to register the client.
On 21 October 2015 at 14:40, Thomas Raehalme
<thomas.raehalme(a)aitiofinland.com
<mailto:thomas.raehalme@aitiofinland.com>> wrote:
If you deploy the same application multiple times for different
customers you could have a configuration template containing all
the common bits and pieces, but have Keycloak generate keys and
secrets when you import the configuration.
Best regards,
Thomas
On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen
<sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote:
Can you elaborate a bit on that?
On 21 October 2015 at 14:29, Thomas Raehalme
<thomas.raehalme(a)aitiofinland.com
<mailto:thomas.raehalme@aitiofinland.com>> wrote:
On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen
<sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote:
I'd like to get import/export done properly. The
addition of being able to add bits and pieces to
import in a directory would be really helpful on
Docker/OpenShift/etc..
I had similar things in mind when I suggested the
re-generation of keys and secrets. You could define a
template which you'd use in a process for new deployments.
Best regards,
Thomas
--
*Thomas Raehalme*
/CTO, teknologiajohtaja/
Mobile +358 40 545 0605 <tel:%2B358%2040%20545%200605>
*Aitio Finland Oy*
Väinönkatu 26 A
40100 JYVÄSKYLÄ, Finland
Tel. +358 10 322 0040 <tel:%2B358%2010%20322%200040>
www.aitiofinland.com <http://www.codecenter.fi>
*Codecenter on nyt Aitio – me kun ei vain koodata!*
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
8:09 a.m.
We already have paginated export of users though?
On 23 October 2015 at 14:22, Marek Posolda <mposolda(a)redhat.com> wrote:
Sorry for late response. Is the scope of this task also to improve
big
long-running export/import ? We already have someone with 75.000 users in
DB.
It seems that export/import is kind of task, which can be easily
"paginated" (Export of user1 - user500 is page1, export of user501-user1000
is page2 etc). We may have the table where you can see the progress of
long-running export/import job and how many users (pages) are already
exported.
Besides that we may support:
- Concurrency (more worker threads. Each worker doing an export of
different page)
- Cluster scalability (Each cluster node takes some pages and helps with
the overall export task)
- Cluster failover (When some cluster node crashes during export job, the
whole job is not canceled but continue on other cluster nodes without
problem)
I've already addressed the concurrency, scalability and failover with the
InfinispanUserSessionInitializer,
which is used at startup for pre-load offline sessions from DB. It's based
on infinispan Distributed Executor service. I think that with some minor
changes (maybe even without them) it can be reused for any long running
"paginatable" job (export/import, sync of big number of users from
federationProvider etc)
Marek
On 21/10/15 14:44, Stian Thorgersen wrote:
I've just added client registration services in 1.6 which should be more
useful for that. There's also a Java library. Basically admins and service
accounts with the role create-client can create new clients, while clients
themselves have permissions to update their config.
You could have a client template that you change the client-id only then
use this endpoint to register the client.
On 21 October 2015 at 14:40, Thomas Raehalme <
thomas.raehalme(a)aitiofinland.com> wrote:
> If you deploy the same application multiple times for different customers
> you could have a configuration template containing all the common bits and
> pieces, but have Keycloak generate keys and secrets when you import the
> configuration.
>
> Best regards,
> Thomas
>
>
> On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen <sthorger(a)redhat.com>
> wrote:
>
>> Can you elaborate a bit on that?
>>
>> On 21 October 2015 at 14:29, Thomas Raehalme <
>> <thomas.raehalme@aitiofinland.com>thomas.raehalme(a)aitiofinland.com>
>> wrote:
>>
>>>
>>>
>>> On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen <
>>> <sthorger@redhat.com>sthorger(a)redhat.com> wrote:
>>>
>>>> I'd like to get import/export done properly. The addition of being
>>>> able to add bits and pieces to import in a directory would be really
>>>> helpful on Docker/OpenShift/etc..
>>>>
>>>
>>> I had similar things in mind when I suggested the re-generation of keys
>>> and secrets. You could define a template which you'd use in a process
for
>>> new deployments.
>>>
>>> Best regards,
>>> Thomas
>>>
>>
>>
>
>
> --
> *Thomas Raehalme*
> *CTO, teknologiajohtaja*
> Mobile +358 40 545 0605 <%2B358%2040%20545%200605>
>
> *Aitio Finland Oy*
> Väinönkatu 26 A
> 40100 JYVÄSKYLÄ, Finland
> Tel. +358 10 322 0040 <%2B358%2010%20322%200040>
> <http://www.codecenter.fi>www.aitiofinland.com
>
> *Codecenter on nyt Aitio – me kun ei vain koodata!*
>
>
_______________________________________________
keycloak-dev mailing
listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
8:11 a.m.
On 10/23/2015 9:09 AM, Stian Thorgersen wrote:
We already have paginated export of users though?
I haven't
started on export. So far, I've done batch import of users,
clients, and identity providers.
On 23 October 2015 at 14:22, Marek Posolda <mposolda(a)redhat.com
<mailto:mposolda@redhat.com>> wrote:
Sorry for late response. Is the scope of this task also to improve
big long-running export/import ? We already have someone with
75.000 users in DB.
It seems that export/import is kind of task, which can be easily
"paginated" (Export of user1 - user500 is page1, export of
user501-user1000 is page2 etc). We may have the table where you
can see the progress of long-running export/import job and how
many users (pages) are already exported.
Besides that we may support:
- Concurrency (more worker threads. Each worker doing an export of
different page)
- Cluster scalability (Each cluster node takes some pages and
helps with the overall export task)
- Cluster failover (When some cluster node crashes during export
job, the whole job is not canceled but continue on other cluster
nodes without problem)
I've already addressed the concurrency, scalability and failover
with the InfinispanUserSessionInitializer, which is used at
startup for pre-load offline sessions from DB. It's based on
infinispan Distributed Executor service. I think that with some
minor changes (maybe even without them) it can be reused for any
long running "paginatable" job (export/import, sync of big number
of users from federationProvider etc)
Marek
On 21/10/15 14:44, Stian Thorgersen wrote:
> I've just added client registration services in 1.6 which should
> be more useful for that. There's also a Java library. Basically
> admins and service accounts with the role create-client can
> create new clients, while clients themselves have permissions to
> update their config.
>
> You could have a client template that you change the client-id
> only then use this endpoint to register the client.
>
> On 21 October 2015 at 14:40, Thomas Raehalme
> <thomas.raehalme(a)aitiofinland.com
> <mailto:thomas.raehalme@aitiofinland.com>> wrote:
>
> If you deploy the same application multiple times for
> different customers you could have a configuration template
> containing all the common bits and pieces, but have Keycloak
> generate keys and secrets when you import the configuration.
>
> Best regards,
> Thomas
>
>
> On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen
> <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote:
>
> Can you elaborate a bit on that?
>
> On 21 October 2015 at 14:29, Thomas Raehalme
> <thomas.raehalme(a)aitiofinland.com
> <mailto:thomas.raehalme@aitiofinland.com>> wrote:
>
>
>
> On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen
> <sthorger(a)redhat.com <mailto:sthorger@redhat.com>>
wrote:
>
> I'd like to get import/export done properly. The
> addition of being able to add bits and pieces to
> import in a directory would be really helpful on
> Docker/OpenShift/etc..
>
>
> I had similar things in mind when I suggested the
> re-generation of keys and secrets. You could define a
> template which you'd use in a process for new
> deployments.
>
> Best regards,
> Thomas
>
>
>
>
>
> --
> *Thomas Raehalme*
> /CTO, teknologiajohtaja/
> Mobile +358 40 545 0605 <tel:%2B358%2040%20545%200605>
>
> *Aitio Finland Oy*
> Väinönkatu 26 A
> 40100 JYVÄSKYLÄ, Finland
> Tel. +358 10 322 0040 <tel:%2B358%2010%20322%200040>
> www.aitiofinland.com <http://www.aitiofinland.com>
>
> *Codecenter on nyt Aitio -- me kun ei vain koodata!*
>
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
8:14 a.m.
The good old import/export stuff does pagination. I don't think the admin
console export you're working on needs pagination.
On 23 October 2015 at 15:11, Stan Silvert <ssilvert(a)redhat.com> wrote:
On 10/23/2015 9:09 AM, Stian Thorgersen wrote:
We already have paginated export of users though?
I haven't started on export. So far, I've done batch import of users,
clients, and identity providers.
On 23 October 2015 at 14:22, Marek Posolda <mposolda(a)redhat.com> wrote:
> Sorry for late response. Is the scope of this task also to improve big
> long-running export/import ? We already have someone with 75.000 users in
> DB.
>
> It seems that export/import is kind of task, which can be easily
> "paginated" (Export of user1 - user500 is page1, export of
user501-user1000
> is page2 etc). We may have the table where you can see the progress of
> long-running export/import job and how many users (pages) are already
> exported.
>
> Besides that we may support:
> - Concurrency (more worker threads. Each worker doing an export of
> different page)
> - Cluster scalability (Each cluster node takes some pages and helps with
> the overall export task)
> - Cluster failover (When some cluster node crashes during export job, the
> whole job is not canceled but continue on other cluster nodes without
> problem)
>
> I've already addressed the concurrency, scalability and failover with the
InfinispanUserSessionInitializer,
> which is used at startup for pre-load offline sessions from DB. It's based
> on infinispan Distributed Executor service. I think that with some minor
> changes (maybe even without them) it can be reused for any long running
> "paginatable" job (export/import, sync of big number of users from
> federationProvider etc)
>
> Marek
>
> On 21/10/15 14:44, Stian Thorgersen wrote:
>
> I've just added client registration services in 1.6 which should be more
> useful for that. There's also a Java library. Basically admins and service
> accounts with the role create-client can create new clients, while clients
> themselves have permissions to update their config.
>
> You could have a client template that you change the client-id only then
> use this endpoint to register the client.
>
> On 21 October 2015 at 14:40, Thomas Raehalme <
> thomas.raehalme(a)aitiofinland.com> wrote:
>
>> If you deploy the same application multiple times for different
>> customers you could have a configuration template containing all the common
>> bits and pieces, but have Keycloak generate keys and secrets when you
>> import the configuration.
>>
>> Best regards,
>> Thomas
>>
>>
>> On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen <sthorger(a)redhat.com>
>> wrote:
>>
>>> Can you elaborate a bit on that?
>>>
>>> On 21 October 2015 at 14:29, Thomas Raehalme <
>>> thomas.raehalme(a)aitiofinland.com> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen
<sthorger(a)redhat.com
>>>> > wrote:
>>>>
>>>>> I'd like to get import/export done properly. The addition of
being
>>>>> able to add bits and pieces to import in a directory would be really
>>>>> helpful on Docker/OpenShift/etc..
>>>>>
>>>>
>>>> I had similar things in mind when I suggested the re-generation of
>>>> keys and secrets. You could define a template which you'd use in a
process
>>>> for new deployments.
>>>>
>>>> Best regards,
>>>> Thomas
>>>>
>>>
>>>
>>
>>
>> --
>> *Thomas Raehalme*
>> *CTO, teknologiajohtaja*
>> Mobile +358 40 545 0605
>>
>> *Aitio Finland Oy*
>> Väinönkatu 26 A
>> 40100 JYVÄSKYLÄ, Finland
>> Tel. +358 10 322 0040
>> www.aitiofinland.com
>>
>> *Codecenter on nyt Aitio – me kun ei vain koodata!*
>>
>>
>
>
> _______________________________________________
> keycloak-dev mailing
listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>
>
_______________________________________________
keycloak-dev mailing
listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
9:37 a.m.
Yes, DirExportProvider provides pagination in separate transactions. But
it's doing everything sequentially on single cluster node (no cluster
awareness). If host crashes after 50.000 users exported, you need to
start again.
In shortcut, what we don't yet have is: concurrency, cluster scalability
+ failover, possibility to see progress of long running task in admin
console
Marek
On 23/10/15 15:09, Stian Thorgersen wrote:
We already have paginated export of users though?
On 23 October 2015 at 14:22, Marek Posolda <mposolda(a)redhat.com
<mailto:mposolda@redhat.com>> wrote:
Sorry for late response. Is the scope of this task also to improve
big long-running export/import ? We already have someone with
75.000 users in DB.
It seems that export/import is kind of task, which can be easily
"paginated" (Export of user1 - user500 is page1, export of
user501-user1000 is page2 etc). We may have the table where you
can see the progress of long-running export/import job and how
many users (pages) are already exported.
Besides that we may support:
- Concurrency (more worker threads. Each worker doing an export of
different page)
- Cluster scalability (Each cluster node takes some pages and
helps with the overall export task)
- Cluster failover (When some cluster node crashes during export
job, the whole job is not canceled but continue on other cluster
nodes without problem)
I've already addressed the concurrency, scalability and failover
with the InfinispanUserSessionInitializer, which is used at
startup for pre-load offline sessions from DB. It's based on
infinispan Distributed Executor service. I think that with some
minor changes (maybe even without them) it can be reused for any
long running "paginatable" job (export/import, sync of big number
of users from federationProvider etc)
Marek
On 21/10/15 14:44, Stian Thorgersen wrote:
> I've just added client registration services in 1.6 which should
> be more useful for that. There's also a Java library. Basically
> admins and service accounts with the role create-client can
> create new clients, while clients themselves have permissions to
> update their config.
>
> You could have a client template that you change the client-id
> only then use this endpoint to register the client.
>
> On 21 October 2015 at 14:40, Thomas Raehalme
> <thomas.raehalme(a)aitiofinland.com
> <mailto:thomas.raehalme@aitiofinland.com>> wrote:
>
> If you deploy the same application multiple times for
> different customers you could have a configuration template
> containing all the common bits and pieces, but have Keycloak
> generate keys and secrets when you import the configuration.
>
> Best regards,
> Thomas
>
>
> On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen
> <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote:
>
> Can you elaborate a bit on that?
>
> On 21 October 2015 at 14:29, Thomas Raehalme
> <thomas.raehalme(a)aitiofinland.com
> <mailto:thomas.raehalme@aitiofinland.com>> wrote:
>
>
>
> On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen
> <sthorger(a)redhat.com <mailto:sthorger@redhat.com>>
wrote:
>
> I'd like to get import/export done properly. The
> addition of being able to add bits and pieces to
> import in a directory would be really helpful on
> Docker/OpenShift/etc..
>
>
> I had similar things in mind when I suggested the
> re-generation of keys and secrets. You could define a
> template which you'd use in a process for new
> deployments.
>
> Best regards,
> Thomas
>
>
>
>
>
> --
> *Thomas Raehalme*
> /CTO, teknologiajohtaja/
> Mobile +358 40 545 0605 <tel:%2B358%2040%20545%200605>
>
> *Aitio Finland Oy*
> Väinönkatu 26 A
> 40100 JYVÄSKYLÄ, Finland
> Tel. +358 10 322 0040 <tel:%2B358%2010%20322%200040>
> www.aitiofinland.com <http://www.aitiofinland.com>
>
> *Codecenter on nyt Aitio – me kun ei vain koodata!*
>
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
3372
days inactive
3374
days old
13 comments
4 participants
participants (4)
-
Marek Posolda -
Stan Silvert -
Stian Thorgersen -
Thomas Raehalme