[keycloak-dev] Export/import implementation

[Marek Posolda]

I am planning to start soon on export/import. If I recall correctly, one 
of the requirements is to export the content of whole DB content 
(including IDs and password hashes) to JSON file, which can then be 
later imported into other DB. This will allow to migrate between 
environments and various DB types (For example from Mongo to MySQL and 
viceversa).

I have some question though

1) I assume that DB should be cleared before full import from JSON file? 
Or do we want to update existing data without deleting the previous 
content? I assume that this is used for migration, so it's not about 
updating but completely delete and recreate existing DB, correct?

2) How to implement it. I can see two approaches

a) Use model API to retrieve content of the DB into JSON file during 
export. Similarly during import use model API to sync objects back from 
JSON into model DB.

b) Add some methods to KeycloakSession interface like:

ObjectNode export();

void import(ObjectNode node);

and implement export/import separately for each model.

Approach (b) might be better for performance as it allows to directly 
use low-level queries specific to JPA, Mongo or other model 
implementations to export/import stuff more effectively in batch, 
however it will require changes in model implementations and probably 
adding more stuff into dependencies. So I am more convinced to use (a). 
Thoughts?


3) How will be export/import triggered?

I think that for security reasons, we want to always export into local 
file with KC server and similarly always import from local file. Is it 
correct? I can see approaches like:

a) Use KC admin console. By default, just "admin" will be able to 
export/import stuff . Data will be always exported/imported into JSON 
file local to server. So it will be possible to trigger export/import 
remotely from admin console, but just use local JSON file. The import 
would be tricky though as import will recreate all data (including admin 
realm and admin user) so it would need to cancel logout sessions 
including the session of admin user who triggered import.

b) Use some script, which will trigger JVM process for export/import 
data. Script can be executed locally from CMD. I can imagine something 
like this (executed from the directory with AS7/wildfly server):

./bin/keycloak-import -Dkeycloak.model=jpa 
-Ddata-to-import=/tmp/file-with-full-kc-data-to-import-which-were-previously-exported.json

Assumption is that distribution contains deployed KC inside 
standalone/deployments/auth-server.war. Script will be able to run JVM 
which will have access to needed KC jars on classpath and access to 
persistence.xml . In AS7/Wildfly environment the persistence.xml bundled 
inside auth-server.war is using datasource though, so this JVM process 
would also need to parse datasource from 
standalone/configuration/standalone.xml as it won't be executed in 
managed AS7/Wildfly environment.

c) Use something similar to approach (b) but execute export/import from 
AS7/Wildfly CLI or admin console. The advantage is that it is triggered 
from the managed (AS7 or Wildfly) so has access to server resources like 
datasource referenced from persistence.xml

Any other possibility I am missing?

Thanks,
Marek