7:42 p.m.
Hello Bill,
I didn't say that using JPA to store event data would lead to bad
performance :-)
it's just that querying and aggregating event data via JPA(QL) is a pain
...
and I didn't want to potentially pull 100k+ events into memory to compute
the stats in Java.
I wanted to keep the processing in the Keycloak JVM as low as possible to
not risk any
OOME scenarios (though one could also use batch fetching here...).
That's the reason why I used a view that uses database specific means to do
so.
With that said recording events with JPA is fine IMHO - though it could be
done asynchronously in the background.
My current approach for computing stats only relies on the already stored
event_entity data,
which of course needs to be persisted then - currently I don't need to
store anything.
But since the event_entity data can be deleted I'd proposed to periodically
compute
aggregates based on those when a day is over. It could then compute stuff
like:
* totalUserCount - how many users do we have in the realm?
* totalLoginCount - how many users did login on that
particular day?
* totalLoginFailedCount - how many logins failed on that day?
* totalLoginBlockedCount - how many failed logins lead to blocked
accounts?
* totalRegisterCount - how many users did register on that day?
* totalAbortedRegistrationsCount - how many users abandonned the
registration?
- per realm per day.
Similar stats could also be held by each keycloak instance in-memory and
shared via infinispan.
Those stats could then also periodically aggregated (every 5 mins) to give
information near real-time.
Regarding the rare usage of this info I'd argue that the dashboard should
actually be the first
page that an administrator sees when logging in to Keycloak (after initial
setup of course) -
I think it gives you a lot more than just a few stats - e.g. the
possibility to login
see patterns and identify potential problems this probably consulted
multiple times a day by multiple
people - especially if one uses a lot of different tenants.
Cheers,
Thomas
2016-05-31 6:15 GMT+02:00 Bill Burke <bburke(a)redhat.com>:
I didn't read whole thread on this: Having a JPA event store
would be bad
performance? Isn't there more than one even per login? That means
multiple DB inserts per login just to gather stats. Stats that would be
looked at rarely (once a day? once a week? once a month?) Just something
to think about.
On 5/29/16 4:52 PM, Thomas Darimont wrote:
Hello group,
a few months ago I raised the feature request "Activity dashboard" in the
Keycloak JIRA.
https://issues.jboss.org/browse/KEYCLOAK-1840
This weekend I gave this a spin and I think I got pretty far with it,
see attached annotated screenshot.
The idea was to leverage the information from the stored event data
to compute some Keycloak usage statistics over time.
My current prototype supports JPA (user / event) storage provider
and works with postgresql but could be adapted to other databases
including MongoDB.
Since I need to compute the usage statistics based on the event data,
events need to be stored and some views (3) need to be defined to
make the data accessible from JPA in a generic fashion.
Since the queries are quite complex I wanted to keep them out
of the code and therefore used named native queries via orm.xml.
The actual queries use some database specific date/time functions
that I wanted to keep out of the code - thus I created views
that could be adapted for each database and provisioned via liquibase.
The view definitions can be found here:
https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e
For MongoDB one could define appropriate aggregation framework pipelines
to express the same query logic.
I basically exposed the data from those views per realm via a newly
introduced AnalyticsProvider interface that is accessible via
KeycloakSession.
Data from this AnalyticsProvider is then exposed as a REST resource called
"DashboardResource".
Data from this REST endpoint is then consumed by the admin frontend in a
new section
called "dashboard".
In the frontend I used basic patternfly components, e.g.: cards & tables:
https://rawgit.com/patternfly/patternfly/master/tests/cards.html
For the heatmap I used http://cal-heatmap.com/#start which is based on
d3js.
There is also an angularjs directive that could be used as well.
https://github.com/shekhargulati/angular-cal-heatmap-directive
The current hacky code can be found here.:
https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dash...
The relevant commit is:
https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2dd...
The code still needs a decent amount of polishing but I wanted to share
this with
you guys first to see if this could make it into Keycloak at some point.
Cheers,
Thomas
_______________________________________________
keycloak-dev mailing
listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev