[keycloak-dev] Batch import/export

[Stian Thorgersen]

We already have paginated export of users though?

On 23 October 2015 at 14:22, Marek Posolda <mposolda at redhat.com> wrote:

> Sorry for late response. Is the scope of this task also to improve big
> long-running export/import ? We already have someone with 75.000 users in
> DB.
>
> It seems that export/import is kind of task, which can be easily
> "paginated" (Export of user1 - user500 is page1, export of user501-user1000
> is page2 etc). We may have the table where you can see the progress of
> long-running export/import job and how many users (pages) are already
> exported.
>
> Besides that we may support:
> - Concurrency (more worker threads. Each worker doing an export of
> different page)
> - Cluster scalability (Each cluster node takes some pages and helps with
> the overall export task)
> - Cluster failover (When some cluster node crashes during export job, the
> whole job is not canceled but continue on other cluster nodes without
> problem)
>
> I've already addressed the concurrency, scalability and failover with the InfinispanUserSessionInitializer,
> which is used at startup for pre-load offline sessions from DB. It's based
> on infinispan Distributed Executor service. I think that with some minor
> changes (maybe even without them) it can be reused for any long running
> "paginatable" job (export/import, sync of big number of users from
> federationProvider etc)
>
> Marek
>
> On 21/10/15 14:44, Stian Thorgersen wrote:
>
> I've just added client registration services in 1.6 which should be more
> useful for that. There's also a Java library. Basically admins and service
> accounts with the role create-client can create new clients, while clients
> themselves have permissions to update their config.
>
> You could have a client template that you change the client-id only then
> use this endpoint to register the client.
>
> On 21 October 2015 at 14:40, Thomas Raehalme <
> thomas.raehalme at aitiofinland.com> wrote:
>
>> If you deploy the same application multiple times for different customers
>> you could have a configuration template containing all the common bits and
>> pieces, but have Keycloak generate keys and secrets when you import the
>> configuration.
>>
>> Best regards,
>> Thomas
>>
>>
>> On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen <sthorger at redhat.com>
>> wrote:
>>
>>> Can you elaborate a bit on that?
>>>
>>> On 21 October 2015 at 14:29, Thomas Raehalme <
>>> <thomas.raehalme at aitiofinland.com>thomas.raehalme at aitiofinland.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen <
>>>> <sthorger at redhat.com>sthorger at redhat.com> wrote:
>>>>
>>>>> I'd like to get import/export done properly. The addition of being
>>>>> able to add bits and pieces to import in a directory would be really
>>>>> helpful on Docker/OpenShift/etc..
>>>>>
>>>>
>>>> I had similar things in mind when I suggested the re-generation of keys
>>>> and secrets. You could define a template which you'd use in a process for
>>>> new deployments.
>>>>
>>>> Best regards,
>>>> Thomas
>>>>
>>>
>>>
>>
>>
>> --
>> *Thomas Raehalme*
>> *CTO, teknologiajohtaja*
>> Mobile +358 40 545 0605 <%2B358%2040%20545%200605>
>>
>> *Aitio Finland Oy*
>> Väinönkatu 26 A
>> 40100 JYVÄSKYLÄ, Finland
>> Tel. +358 10 322 0040 <%2B358%2010%20322%200040>
>> <http://www.codecenter.fi>www.aitiofinland.com
>>
>> *Codecenter on nyt Aitio – me kun ei vain koodata!*
>>
>>
>
>
> _______________________________________________
> keycloak-dev mailing listkeycloak-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/keycloak-dev/attachments/20151023/31842d9a/attachment-0001.html