[keycloak-dev] Export/import implementation

[Bill Burke]

On 4/28/2014 3:27 AM, Marek Posolda wrote:
> I am planning to start soon on export/import. If I recall correctly, one
> of the requirements is to export the content of whole DB content
> (including IDs and password hashes) to JSON file, which can then be
> later imported into other DB. This will allow to migrate between
> environments and various DB types (For example from Mongo to MySQL and
> viceversa).
>

IMO, a full export (of credentials) should require a secret given by the 
admin that will be used to encrypt the export.  The export should only 
be saved locally to disk and not available over the network.

> I have some question though
>
> 1) I assume that DB should be cleared before full import from JSON file?
> Or do we want to update existing data without deleting the previous
> content? I assume that this is used for migration, so it's not about
> updating but completely delete and recreate existing DB, correct?
>
> 2) How to implement it. I can see two approaches
>
> a) Use model API to retrieve content of the DB into JSON file during
> export. Similarly during import use model API to sync objects back from
> JSON into model DB.
>
> b) Add some methods to KeycloakSession interface like:
>
> ObjectNode export();
>
> void import(ObjectNode node);
>
> and implement export/import separately for each model.
>
> Approach (b) might be better for performance as it allows to directly
> use low-level queries specific to JPA, Mongo or other model
> implementations to export/import stuff more effectively in batch,
> however it will require changes in model implementations and probably
> adding more stuff into dependencies. So I am more convinced to use (a).
> Thoughts?
>

"a", IMO.  Easier to maintain.

>
> 3) How will be export/import triggered?
>
> I think that for security reasons, we want to always export into local
> file with KC server and similarly always import from local file. Is it
> correct? I can see approaches like:
>

You can already import a full json description from the admin console.

> a) Use KC admin console. By default, just "admin" will be able to
> export/import stuff . Data will be always exported/imported into JSON
> file local to server. So it will be possible to trigger export/import
> remotely from admin console, but just use local JSON file. The import
> would be tricky though as import will recreate all data (including admin
> realm and admin user) so it would need to cancel logout sessions
> including the session of admin user who triggered import.
>
> b) Use some script, which will trigger JVM process for export/import
> data. Script can be executed locally from CMD. I can imagine something
> like this (executed from the directory with AS7/wildfly server):
>
> ./bin/keycloak-import -Dkeycloak.model=jpa
> -Ddata-to-import=/tmp/file-with-full-kc-data-to-import-which-were-previously-exported.json
>
> Assumption is that distribution contains deployed KC inside
> standalone/deployments/auth-server.war. Script will be able to run JVM
> which will have access to needed KC jars on classpath and access to
> persistence.xml . In AS7/Wildfly environment the persistence.xml bundled
> inside auth-server.war is using datasource though, so this JVM process
> would also need to parse datasource from
> standalone/configuration/standalone.xml as it won't be executed in
> managed AS7/Wildfly environment.
>
> c) Use something similar to approach (b) but execute export/import from
> AS7/Wildfly CLI or admin console. The advantage is that it is triggered
> from the managed (AS7 or Wildfly) so has access to server resources like
> datasource referenced from persistence.xml
>

I don't see any reason you couldn't trigger an import from the admin 
console.  Especially if we require a secret used to encrypt the export.


-- 
Bill Burke
JBoss, a division of Red Hat
http://bill.burkecentral.com