[keycloak-dev] Batch import/export

[Stan Silvert]

On 10/23/2015 9:09 AM, Stian Thorgersen wrote:
> We already have paginated export of users though?
I haven't started on export.  So far, I've done batch import of users, 
clients, and identity providers.
>
> On 23 October 2015 at 14:22, Marek Posolda <mposolda at redhat.com 
> <mailto:mposolda at redhat.com>> wrote:
>
>     Sorry for late response. Is the scope of this task also to improve
>     big long-running export/import ? We already have someone with
>     75.000 users in DB.
>
>     It seems that export/import is kind of task, which can be easily
>     "paginated" (Export of user1 - user500 is page1, export of
>     user501-user1000 is page2 etc). We may have the table where you
>     can see the progress of long-running export/import job and how
>     many users (pages) are already exported.
>
>     Besides that we may support:
>     - Concurrency (more worker threads. Each worker doing an export of
>     different page)
>     - Cluster scalability (Each cluster node takes some pages and
>     helps with the overall export task)
>     - Cluster failover (When some cluster node crashes during export
>     job, the whole job is not canceled but continue on other cluster
>     nodes without problem)
>
>     I've already addressed the concurrency, scalability and failover
>     with the InfinispanUserSessionInitializer, which is used at
>     startup for pre-load offline sessions from DB. It's based on
>     infinispan Distributed Executor service. I think that with some
>     minor changes (maybe even without them) it can be reused for any
>     long running "paginatable" job (export/import, sync of big number
>     of users from federationProvider etc)
>
>     Marek
>
>     On 21/10/15 14:44, Stian Thorgersen wrote:
>>     I've just added client registration services in 1.6 which should
>>     be more useful for that. There's also a Java library. Basically
>>     admins and service accounts with the role create-client can
>>     create new clients, while clients themselves have permissions to
>>     update their config.
>>
>>     You could have a client template that you change the client-id
>>     only then use this endpoint to register the client.
>>
>>     On 21 October 2015 at 14:40, Thomas Raehalme
>>     <thomas.raehalme at aitiofinland.com
>>     <mailto:thomas.raehalme at aitiofinland.com>> wrote:
>>
>>         If you deploy the same application multiple times for
>>         different customers you could have a configuration template
>>         containing all the common bits and pieces, but have Keycloak
>>         generate keys and secrets when you import the configuration.
>>
>>         Best regards,
>>         Thomas
>>
>>
>>         On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen
>>         <sthorger at redhat.com <mailto:sthorger at redhat.com>> wrote:
>>
>>             Can you elaborate a bit on that?
>>
>>             On 21 October 2015 at 14:29, Thomas Raehalme
>>             <thomas.raehalme at aitiofinland.com
>>             <mailto:thomas.raehalme at aitiofinland.com>> wrote:
>>
>>
>>
>>                 On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen
>>                 <sthorger at redhat.com <mailto:sthorger at redhat.com>> wrote:
>>
>>                     I'd like to get import/export done properly. The
>>                     addition of being able to add bits and pieces to
>>                     import in a directory would be really helpful on
>>                     Docker/OpenShift/etc..
>>
>>
>>                 I had similar things in mind when I suggested the
>>                 re-generation of keys and secrets. You could define a
>>                 template which you'd use in a process for new
>>                 deployments.
>>
>>                 Best regards,
>>                 Thomas
>>
>>
>>
>>
>>
>>         -- 
>>         *Thomas Raehalme*
>>         /CTO, teknologiajohtaja/
>>         Mobile +358 40 545 0605 <tel:%2B358%2040%20545%200605>
>>
>>         *Aitio Finland Oy*
>>         Väinönkatu 26 A
>>         40100 JYVÄSKYLÄ, Finland
>>         Tel. +358 10 322 0040 <tel:%2B358%2010%20322%200040>
>>         www.aitiofinland.com <http://www.aitiofinland.com>
>>
>>         *Codecenter on nyt Aitio -- me kun ei vain koodata!*
>>
>>
>>
>>
>>     _______________________________________________
>>     keycloak-dev mailing list
>>     keycloak-dev at lists.jboss.org  <mailto:keycloak-dev at lists.jboss.org>
>>     https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151023/d118c850/attachment.html