[keycloak-dev] Batch import/export

[Marek Posolda]

Sorry for late response. Is the scope of this task also to improve big 
long-running export/import ? We already have someone with 75.000 users 
in DB.

It seems that export/import is kind of task, which can be easily 
"paginated" (Export of user1 - user500 is page1, export of 
user501-user1000 is page2 etc). We may have the table where you can see 
the progress of long-running export/import job and how many users 
(pages) are already exported.

Besides that we may support:
- Concurrency (more worker threads. Each worker doing an export of 
different page)
- Cluster scalability (Each cluster node takes some pages and helps with 
the overall export task)
- Cluster failover (When some cluster node crashes during export job, 
the whole job is not canceled but continue on other cluster nodes 
without problem)

I've already addressed the concurrency, scalability and failover with 
the InfinispanUserSessionInitializer, which is used at startup for 
pre-load offline sessions from DB. It's based on infinispan Distributed 
Executor service. I think that with some minor changes (maybe even 
without them) it can be reused for any long running "paginatable" job 
(export/import, sync of big number of users from federationProvider etc)

Marek

On 21/10/15 14:44, Stian Thorgersen wrote:
> I've just added client registration services in 1.6 which should be 
> more useful for that. There's also a Java library. Basically admins 
> and service accounts with the role create-client can create new 
> clients, while clients themselves have permissions to update their 
> config.
>
> You could have a client template that you change the client-id only 
> then use this endpoint to register the client.
>
> On 21 October 2015 at 14:40, Thomas Raehalme 
> <thomas.raehalme at aitiofinland.com 
> <mailto:thomas.raehalme at aitiofinland.com>> wrote:
>
>     If you deploy the same application multiple times for different
>     customers you could have a configuration template containing all
>     the common bits and pieces, but have Keycloak generate keys and
>     secrets when you import the configuration.
>
>     Best regards,
>     Thomas
>
>
>     On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen
>     <sthorger at redhat.com <mailto:sthorger at redhat.com>> wrote:
>
>         Can you elaborate a bit on that?
>
>         On 21 October 2015 at 14:29, Thomas Raehalme
>         <thomas.raehalme at aitiofinland.com
>         <mailto:thomas.raehalme at aitiofinland.com>> wrote:
>
>
>
>             On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen
>             <sthorger at redhat.com <mailto:sthorger at redhat.com>> wrote:
>
>                 I'd like to get import/export done properly. The
>                 addition of being able to add bits and pieces to
>                 import in a directory would be really helpful on
>                 Docker/OpenShift/etc..
>
>
>             I had similar things in mind when I suggested the
>             re-generation of keys and secrets. You could define a
>             template which you'd use in a process for new deployments.
>
>             Best regards,
>             Thomas
>
>
>
>
>
>     -- 
>     *Thomas Raehalme*
>     /CTO, teknologiajohtaja/
>     Mobile +358 40 545 0605 <tel:%2B358%2040%20545%200605>
>
>     *Aitio Finland Oy*
>     Väinönkatu 26 A
>     40100 JYVÄSKYLÄ, Finland
>     Tel. +358 10 322 0040 <tel:%2B358%2010%20322%200040>
>     www.aitiofinland.com <http://www.codecenter.fi>
>
>     *Codecenter on nyt Aitio – me kun ei vain koodata!*
>
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151023/cbee520c/attachment.html