[keycloak-dev] Online Export

[Marko Strukelj]

I've been working on online Export functionality for Admin Console
(RHSSO-402).

Primary use case is to get core realm configuration with identity providers
and user storage, but no users, and optionally also excluding clients,
roles, and groups. As opposed to offline export (at startup time), which
can serve as a backup to restore the server, the idea of online export is
to help gather server configuration for troubleshooting so you can
reproduce / diagnose issues.

In terms of implementation the idea was first to just trigger the same
logic that creates a single JSON export during offline export, just filter
it to have less content, and no secrets. Then one would ssh to the server
to retrieve the file. But, that's a complicated workflow, especially when
dealing with a cluster of servers - how to know which server performed the
export?

A more usable implementation would be to download the exported JSON through
Admin Console. That does return a lot of information through a simple REST
request - anybody sees any issues here?

Another thing is to be careful that triggering an export does not slow or
block processing of other requests to the server. Current implementation
performs single file export within a single UserTransaction. With no users,
groups, roles, and clients there should be no issues here. But if exporting
many thousands of clients, that might interfere with normal operation of
the server.

One idea how to prevent that is to hardcode a fixed limit on instance count
for groups, roles, and clients. Export page would display client count next
to toggle button for exporting of client, and analogous for groups, and
roles. If count is greater than let's say 10k, the toggle for that
inclusion would be disabled. That will require an extra rest endpoint for
gathering these counts. To consistently enforce the limits the check would
have to be performed inside Admin REST export call as well.

Such implementation is more complicated but the whole workflow to get to
exported JSON is much better.

Any thoughts on such an approach?