Re: [keycloak-dev] Batch import/export

We already have paginated export of users though?

On 23 October 2015 at 14:22, Marek Posolda <mposolda(a)redhat.com <mailto:mposolda@redhat.com>> wrote: Sorry for late response. Is the scope of this task also to improve big long-running export/import ? We already have someone with 75.000 users in DB. It seems that export/import is kind of task, which can be easily "paginated" (Export of user1 - user500 is page1, export of user501-user1000 is page2 etc). We may have the table where you can see the progress of long-running export/import job and how many users (pages) are already exported. Besides that we may support: - Concurrency (more worker threads. Each worker doing an export of different page) - Cluster scalability (Each cluster node takes some pages and helps with the overall export task) - Cluster failover (When some cluster node crashes during export job, the whole job is not canceled but continue on other cluster nodes without problem) I've already addressed the concurrency, scalability and failover with the InfinispanUserSessionInitializer, which is used at startup for pre-load offline sessions from DB. It's based on infinispan Distributed Executor service. I think that with some minor changes (maybe even without them) it can be reused for any long running "paginatable" job (export/import, sync of big number of users from federationProvider etc) Marek On 21/10/15 14:44, Stian Thorgersen wrote: > I've just added client registration services in 1.6 which should > be more useful for that. There's also a Java library. Basically > admins and service accounts with the role create-client can > create new clients, while clients themselves have permissions to > update their config. > > You could have a client template that you change the client-id > only then use this endpoint to register the client. > > On 21 October 2015 at 14:40, Thomas Raehalme > <thomas.raehalme(a)aitiofinland.com > <mailto:thomas.raehalme@aitiofinland.com>> wrote: > > If you deploy the same application multiple times for > different customers you could have a configuration template > containing all the common bits and pieces, but have Keycloak > generate keys and secrets when you import the configuration. > > Best regards, > Thomas > > > On Wed, Oct 21, 2015 at 3:33 PM, Stian Thorgersen > <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote: > > Can you elaborate a bit on that? > > On 21 October 2015 at 14:29, Thomas Raehalme > <thomas.raehalme(a)aitiofinland.com > <mailto:thomas.raehalme@aitiofinland.com>> wrote: > > > > On Wed, Oct 21, 2015 at 3:27 PM, Stian Thorgersen > <sthorger(a)redhat.com <mailto:sthorger@redhat.com>> wrote: > > I'd like to get import/export done properly. The > addition of being able to add bits and pieces to > import in a directory would be really helpful on > Docker/OpenShift/etc.. > > > I had similar things in mind when I suggested the > re-generation of keys and secrets. You could define a > template which you'd use in a process for new > deployments. > > Best regards, > Thomas > > > > > > -- > *Thomas Raehalme* > /CTO, teknologiajohtaja/ > Mobile +358 40 545 0605 <tel:%2B358%2040%20545%200605> > > *Aitio Finland Oy* > Väinönkatu 26 A > 40100 JYVÄSKYLÄ, Finland > Tel. +358 10 322 0040 <tel:%2B358%2010%20322%200040> > www.aitiofinland.com <http://www.aitiofinland.com> > > *Codecenter on nyt Aitio -- me kun ei vain koodata!* > > > > > _______________________________________________ > keycloak-dev mailing list > keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/keycloak-dev _______________________________________________ keycloak-dev mailing list keycloak-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/keycloak-dev