[keycloak-dev] Export/import into representations

[Stian Thorgersen]

Great stuff Marek

Some comments inline

----- Original Message -----
> From: "Marek Posolda" <mposolda at redhat.com>
> To: keycloak-dev at lists.jboss.org
> Sent: Tuesday, 15 July, 2014 3:50:32 PM
> Subject: [keycloak-dev] Export/import into representations
> 
> I've sent PR https://github.com/keycloak/keycloak/pull/523 for
> export/import with same representation as admin realm.
> 
> - It's still possible to export into encrypted ZIP file (default
> possibility), into directory or all realms into single JSON file.
> 
> - With export to directory, data of each realm are exported into
> separate file and users are also exported into separate file. It's
> configurable how many users will be in single file (By default it's 5000
> users per file) and it's using separate transaction per each file (set
> of users). So for example if you have realm with 20.000 users the realm
> will be exported into 5 separate files (1 file like "demo-realm.json"
> for realm data and 4 files like "demo-users-0.json", "demo-users-1.json"
> etc for users).
> 
> - Export to encrypted ZIP is quite similar like export to directory
> (realms and users in separate files inside ZIP), but all data are
> canonicalized. This is default possibility and is intended for
> production use
> 
> - Export into single JSON file allows to export all realms including
> users into single JSON file (array of RealmRepresentations). This is
> intended to be used mainly for development
> 
> - I've moved some code from "services" module as from there it's not
> available to export/import. I've moved class ModelToRepresentation to
> model/api and I've also extracted some code from managers (RealmManager,
> ApplicationManager, OAuthClientManager, RolesManager) to
> RepresentationToModel class.
> 
> - I've moved tests to testsuite/integration . Right now there is
> ExportImportTest, which tests all 3 providers (zip, dir, singleFile).
> 
> - Question: Is it plan to move some manager classes like RealmManager,
> ApplicationManager, OAuthClientManager and RolesManager to model/api as
> well? These don't have dependencies on other stuff in "services" and it
> will be useful for stuff like export/import to have them available. For
> example export/import may need the code for setup masterAdmin
> application (as if I import realm, I've need to check if particular
> master application like "demo-realm" exists in master realm and create
> it if doesn't, so I've needed to move some related code from
> RealmManager for setup this too).

+1

> 
> - Export/import is still checked at server startup with system properties.
> Question: How big priority it is to support export from admin console?
> TBH it shouldn't be hard to add it but it seems to me that this option
> is useful more for development than production as during export from
> admin console could be realm data edited in the meantime (For example if
> some other user registers himself in the meantime when admin triggered
> export). The solution might be JAX-RS or Http Filter, which will refuse
> request to realm when export is in progress. Do we want to go that way?
> Maybe just possibility to export current realm to single JSON file is
> sufficient in admin console?

It could be nice to be able to export realms through the console. For exporting realm we could disable any calls to admin console to make sure realms don't change. For users we could disable account mngmt, required actions, registrations, and other things that'll do writes, while still allowing users to login etc.

LDAP/Sync is much higher priority IMO though. Probably best to post-pone this to later, or even wait until/if we get requests for it.

> 
> Let me know if you have some other ideas/proposals
> 
> Thanks,
> Marek
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>