[keycloak-dev] Using export and import for backup and restore

[Sebastian Laskawiec]

On Mon, Sep 30, 2019 at 3:31 PM Stian Thorgersen <sthorger at redhat.com>
wrote:

> Export/import using JSON has a few significant disadvantages:
>
> * It's very very slow
> * It can not provide a consistent snapshot unless all writes are stopped
> during the export
>

I believe any of those two would be a problem.

In the Operator use case, all modifications should be made through the
Operator. That means, we could implement a simple Mutex and prevent the
Operator from modifying anything until the backup is complete.


>
> With that regards I very much doubt it would be the ideal solution.
>

The more I'm thinking about it, the more I'm convinced that's using
export/import gives us the most flexibility.

At first, we could simplify basic configuration of a Realm (and other CRDs
in the future as well). We would expose only basic settings and if anyone
wants to configure every small detail, he would need to prepare a full JSON
and restore it - just like restoring a backup - the same mechanism. If
there are some guarantees (are there any?) about the structure of the JSON
backup, we could use it for complicated migration process - like
Integreately, where they need to migrate off the old Keycloak Operator to a
new one. Also, the structure remains of the JSON file remains compatible
across Keycloak/RHSSO versions, we could use it for migrating our customers
from Keycloak to RHSSO. Finally, this solution doesn't tie us up to a
particular database and its version. During an upgrade, we can wipe the
database up and just restore Keycloak from a JSON file.


>
> One question though can't backup be performed at the DB level, and then be
> a requirement on the DB operator rather than the Keycloak operator?
>

That's another option we consider. We could use Volume Snapshots [1] and
just backup the whole Postgresql data directory. This seems to be the
fastest option and I believe, the Integreately Team tried that before
(Peter, David - perhaps you could tell us more about it). However, it ties
us up to Postgresql in a specific version (as far as I know there are no
guarantees about migrating the data directory between Postgresql versions).
My intuition tells me, this will be a problem in long-term.

[1]
https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/


>
> On Mon, 30 Sep 2019 at 15:22, Sebastian Laskawiec <slaskawi at redhat.com>
> wrote:
>
>> Hey,
>>
>> In the next few days we'll be looking into implementing backup and restore
>> functionality for the Keycloak Operator. One of the options we are
>> considering, is using an export/import functionality. An Operator could
>> export all realms into a JSON file and put it somewhere in a Persistent
>> Volume.
>>
>> I was wondering, what do you think about this approach? Are there any
>> guarantees around export/import functionality (especially with the regards
>> to its format)? Also, would it work for exporting JSON file from Keycloak
>> and importing it to RHSSO?
>>
>> Thanks,
>> Sebastian
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>