3:52 p.m.
Hello group,
a few months ago I raised the feature request "Activity dashboard" in the
Keycloak JIRA.
https://issues.jboss.org/browse/KEYCLOAK-1840
This weekend I gave this a spin and I think I got pretty far with it,
see attached annotated screenshot.
The idea was to leverage the information from the stored event data
to compute some Keycloak usage statistics over time.
My current prototype supports JPA (user / event) storage provider
and works with postgresql but could be adapted to other databases including
MongoDB.
Since I need to compute the usage statistics based on the event data,
events need to be stored and some views (3) need to be defined to
make the data accessible from JPA in a generic fashion.
Since the queries are quite complex I wanted to keep them out
of the code and therefore used named native queries via orm.xml.
The actual queries use some database specific date/time functions
that I wanted to keep out of the code - thus I created views
that could be adapted for each database and provisioned via liquibase.
The view definitions can be found here:
https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e
For MongoDB one could define appropriate aggregation framework pipelines
to express the same query logic.
I basically exposed the data from those views per realm via a newly
introduced AnalyticsProvider interface that is accessible via
KeycloakSession.
Data from this AnalyticsProvider is then exposed as a REST resource called
"DashboardResource".
Data from this REST endpoint is then consumed by the admin frontend in a
new section
called "dashboard".
In the frontend I used basic patternfly components, e.g.: cards & tables:
https://rawgit.com/patternfly/patternfly/master/tests/cards.html
For the heatmap I used http://cal-heatmap.com/#start which is based on d3js.
There is also an angularjs directive that could be used as well.
https://github.com/shekhargulati/angular-cal-heatmap-directive
The current hacky code can be found here.:
https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dash...
The relevant commit is:
https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2dd...
The code still needs a decent amount of polishing but I wanted to share
this with
you guys first to see if this could make it into Keycloak at some point.
Cheers,
Thomas
12:47 a.m.
That's really cool and would be great to have this added to Keycloak.
Some questions/comments:
* What's the (+) under 7 total users?
* What's the purpose of "Logins along the year" - it looks cool, but
I'm
not sure how it'd be used. It would also require storing events for the
whole year.
* I'm not keen on having db specific views. We already support quite a few
dbs so maintaining these would be painful. Would be better to delegate this
to Hibernate if possible and use ejq queries.
* Logins/Registrations should display date and time. At least if date is
not today.
On 29 May 2016 at 22:52, Thomas Darimont <thomas.darimont(a)googlemail.com>
wrote:
Hello group,
a few months ago I raised the feature request "Activity dashboard" in the
Keycloak JIRA.
https://issues.jboss.org/browse/KEYCLOAK-1840
This weekend I gave this a spin and I think I got pretty far with it,
see attached annotated screenshot.
The idea was to leverage the information from the stored event data
to compute some Keycloak usage statistics over time.
My current prototype supports JPA (user / event) storage provider
and works with postgresql but could be adapted to other databases
including MongoDB.
Since I need to compute the usage statistics based on the event data,
events need to be stored and some views (3) need to be defined to
make the data accessible from JPA in a generic fashion.
Since the queries are quite complex I wanted to keep them out
of the code and therefore used named native queries via orm.xml.
The actual queries use some database specific date/time functions
that I wanted to keep out of the code - thus I created views
that could be adapted for each database and provisioned via liquibase.
The view definitions can be found here:
https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e
For MongoDB one could define appropriate aggregation framework pipelines
to express the same query logic.
I basically exposed the data from those views per realm via a newly
introduced AnalyticsProvider interface that is accessible via
KeycloakSession.
Data from this AnalyticsProvider is then exposed as a REST resource called
"DashboardResource".
Data from this REST endpoint is then consumed by the admin frontend in a
new section
called "dashboard".
In the frontend I used basic patternfly components, e.g.: cards & tables:
https://rawgit.com/patternfly/patternfly/master/tests/cards.html
For the heatmap I used http://cal-heatmap.com/#start which is based on
d3js.
There is also an angularjs directive that could be used as well.
https://github.com/shekhargulati/angular-cal-heatmap-directive
The current hacky code can be found here.:
https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dash...
The relevant commit is:
https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2dd...
The code still needs a decent amount of polishing but I wanted to share
this with
you guys first to see if this could make it into Keycloak at some point.
Cheers,
Thomas
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
2:51 a.m.
* What's the (+) under 7 total users?
-> currently it's only a placeholder but the idea is to link to the "create
new user" page
* What's the purpose of "Logins along the year" - it looks cool, but
I'm
not sure how it'd be used.
It would also require storing events for the whole year.
-> This heatmap gives you an idea of the overall realm usage over the year
at a glance - which allows
to recognize patterns visually if the thresholds are calibrated accordingly.
It could also be used to identify login problems e.g. after rolling out a
new version - fewer logins than before
could indicate problems for some users.
* I'm not keen on having db specific views. We already support quite a few
dbs so maintaining these would be painful.
Would be better to delegate this to Hibernate if possible and use ejq
queries.
-> One could of course replace the fews with simple more simple and generic
queries that could also
be expressed via jpaql but this would require some processing within the
keycloak server.
At least for this prototype I wanted to keep the amount of code small.
On the other hand view definitions for each database allow for optimal
performance if you
need to compute statistics / summaries after the fact from events.
One could also compute the statistics eagerly e.g. update with each login
(via custom EventListener).
An alternative approach for computing summary / login statistics would be
to use some kind of approximation mechanism.
E.g. instead of computing the summary from the events one could also use a
sketching data structure like a count min-sketch
that is updated with each login (via custom EventListener) with an
appropriately configured accuracy (e.g. 99%) that work with
a fixed amount of memory.
* Logins/Registrations should display date and time. At least if date is
not today.
-> Date is displayed if date is not the same day ;-)
Some additional remarks:
-> The lower line of the cards in the upper area are currently mocked.
In the login card the "red 4 | 1" is meant to indicate 4 failed logins
and 1 login
that lead to a blocked account.
The "red 1" below the registrations card should indicate 1 failed
registration
attempt (e.g. something wrong in the server side). I can also imagine an
indicator for "aborted" registration attempts.
-> The "latest logins" as well the "New registrations" should
actually be
right next to each other instead of below each other.
-> The REST endpoint, currently called DashboardResource, could also be
exposed
in a more generic form like "AnalyticsResource" which could then offer
various time series and summaries as JSON output for consumption by other
tools like nagios.
2016-05-30 7:47 GMT+02:00 Stian Thorgersen <sthorger(a)redhat.com>:
That's really cool and would be great to have this added to
Keycloak.
Some questions/comments:
* What's the (+) under 7 total users?
* What's the purpose of "Logins along the year" - it looks cool, but
I'm
not sure how it'd be used. It would also require storing events for the
whole year.
* I'm not keen on having db specific views. We already support quite a few
dbs so maintaining these would be painful. Would be better to delegate this
to Hibernate if possible and use ejq queries.
* Logins/Registrations should display date and time. At least if date is
not today.
On 29 May 2016 at 22:52, Thomas Darimont <thomas.darimont(a)googlemail.com>
wrote:
> Hello group,
>
> a few months ago I raised the feature request "Activity dashboard" in the
> Keycloak JIRA.
> https://issues.jboss.org/browse/KEYCLOAK-1840
>
> This weekend I gave this a spin and I think I got pretty far with it,
> see attached annotated screenshot.
>
> The idea was to leverage the information from the stored event data
> to compute some Keycloak usage statistics over time.
> My current prototype supports JPA (user / event) storage provider
> and works with postgresql but could be adapted to other databases
> including MongoDB.
>
> Since I need to compute the usage statistics based on the event data,
> events need to be stored and some views (3) need to be defined to
> make the data accessible from JPA in a generic fashion.
>
> Since the queries are quite complex I wanted to keep them out
> of the code and therefore used named native queries via orm.xml.
> The actual queries use some database specific date/time functions
> that I wanted to keep out of the code - thus I created views
> that could be adapted for each database and provisioned via liquibase.
>
> The view definitions can be found here:
> https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e
>
> For MongoDB one could define appropriate aggregation framework pipelines
> to express the same query logic.
>
> I basically exposed the data from those views per realm via a newly
> introduced AnalyticsProvider interface that is accessible via
> KeycloakSession.
>
> Data from this AnalyticsProvider is then exposed as a REST resource
> called "DashboardResource".
> Data from this REST endpoint is then consumed by the admin frontend in a
> new section
> called "dashboard".
>
> In the frontend I used basic patternfly components, e.g.: cards & tables:
> https://rawgit.com/patternfly/patternfly/master/tests/cards.html
>
> For the heatmap I used http://cal-heatmap.com/#start which is based on
> d3js.
> There is also an angularjs directive that could be used as well.
> https://github.com/shekhargulati/angular-cal-heatmap-directive
>
> The current hacky code can be found here.:
>
> https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dash...
>
> The relevant commit is:
>
>
https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2dd...
>
> The code still needs a decent amount of polishing but I wanted to share
> this with
> you guys first to see if this could make it into Keycloak at some point.
>
> Cheers,
> Thomas
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
3:56 a.m.
One worry here is that this is pretty much just a rip off from the Auth0
dashboard. That's not ideal IMO and we should have our own designed, maybe
we should include UXP guys from Red Hat to do that. If we list the details
that we want to display they can then design one for us.
On 30 May 2016 at 09:51, Thomas Darimont <thomas.darimont(a)googlemail.com>
wrote:
* What's the (+) under 7 total users?
-> currently it's only a placeholder but the idea is to link to the
"create new user" page
That wasn't clear to me at least. Not sure we need a link to create user
from the dashboard.
* What's the purpose of "Logins along the year" - it looks cool, but
I'm
not sure how it'd be used.
It would also require storing events for the whole year.
-> This heatmap gives you an idea of the overall realm usage over the year
at a glance - which allows
to recognize patterns visually if the thresholds are calibrated
accordingly.
It could also be used to identify login problems e.g. after rolling out a
new version - fewer logins than before
could indicate problems for some users.
Wouldn't a simple graph do the same? In either case you could only display
as much data as available in the database, so it would be good if only
available months are displayed.
* I'm not keen on having db specific views. We already support quite a few
dbs so maintaining these would be painful.
Would be better to delegate this to Hibernate if possible and use ejq
queries.
-> One could of course replace the fews with simple more simple and
generic queries that could also
be expressed via jpaql but this would require some processing within the
keycloak server.
At least for this prototype I wanted to keep the amount of code small.
On the other hand view definitions for each database allow for optimal
performance if you
need to compute statistics / summaries after the fact from events.
One could also compute the statistics eagerly e.g. update with each login
(via custom EventListener).
An alternative approach for computing summary / login statistics would be
to use some kind of approximation mechanism.
E.g. instead of computing the summary from the events one could also use a
sketching data structure like a count min-sketch
that is updated with each login (via custom EventListener) with an
appropriately configured accuracy (e.g. 99%) that work with
a fixed amount of memory.
A simple implementation based on events is probably good enough at least
initially. I appreciate the fact that it may be possible to tweak views for
each database, but we don't have the resources to do that. We already
support PostgreSQL, MySQL, MariaDB, Sybase, MSSQL, ..... For someone to
write specific views for each database is time-consuming and it also needs
to be maintained.
Having to persist additional things on top of events is also not ideal.
Sounds like an approximation or an alternative listener would have to do
that.
IMO performance of logins are much more important than the performance of
the dashboard.
* Logins/Registrations should display date and time. At least if date is
not today.
-> Date is displayed if date is not the same day ;-)
Some additional remarks:
-> The lower line of the cards in the upper area are currently mocked.
In the login card the "red 4 | 1" is meant to indicate 4 failed logins
and 1 login
that lead to a blocked account.
The "red 1" below the registrations card should indicate 1 failed
registration
attempt (e.g. something wrong in the server side). I can also imagine an
indicator for "aborted" registration attempts.
That wasn't clear to me. Failed logins was clear, but not the blocked
account. You could actually argue that blocked accounts should be red as
they are "worse" than a single failed login.
-> The "latest logins" as well the "New registrations" should
actually be
right next to each other instead of below each other.
-> The REST endpoint, currently called DashboardResource, could also be
exposed
in a more generic form like "AnalyticsResource" which could then offer
various time series and summaries as JSON output for consumption by
other tools like nagios.
2016-05-30 7:47 GMT+02:00 Stian Thorgersen <sthorger(a)redhat.com>:
> That's really cool and would be great to have this added to Keycloak.
>
> Some questions/comments:
>
> * What's the (+) under 7 total users?
> * What's the purpose of "Logins along the year" - it looks cool, but
I'm
> not sure how it'd be used. It would also require storing events for the
> whole year.
> * I'm not keen on having db specific views. We already support quite a
> few dbs so maintaining these would be painful. Would be better to delegate
> this to Hibernate if possible and use ejq queries.
> * Logins/Registrations should display date and time. At least if date is
> not today.
>
> On 29 May 2016 at 22:52, Thomas Darimont <thomas.darimont(a)googlemail.com>
> wrote:
>
>> Hello group,
>>
>> a few months ago I raised the feature request "Activity dashboard" in
>> the Keycloak JIRA.
>> https://issues.jboss.org/browse/KEYCLOAK-1840
>>
>> This weekend I gave this a spin and I think I got pretty far with it,
>> see attached annotated screenshot.
>>
>> The idea was to leverage the information from the stored event data
>> to compute some Keycloak usage statistics over time.
>> My current prototype supports JPA (user / event) storage provider
>> and works with postgresql but could be adapted to other databases
>> including MongoDB.
>>
>> Since I need to compute the usage statistics based on the event data,
>> events need to be stored and some views (3) need to be defined to
>> make the data accessible from JPA in a generic fashion.
>>
>> Since the queries are quite complex I wanted to keep them out
>> of the code and therefore used named native queries via orm.xml.
>> The actual queries use some database specific date/time functions
>> that I wanted to keep out of the code - thus I created views
>> that could be adapted for each database and provisioned via liquibase.
>>
>> The view definitions can be found here:
>> https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e
>>
>> For MongoDB one could define appropriate aggregation framework pipelines
>> to express the same query logic.
>>
>> I basically exposed the data from those views per realm via a newly
>> introduced AnalyticsProvider interface that is accessible via
>> KeycloakSession.
>>
>> Data from this AnalyticsProvider is then exposed as a REST resource
>> called "DashboardResource".
>> Data from this REST endpoint is then consumed by the admin frontend in a
>> new section
>> called "dashboard".
>>
>> In the frontend I used basic patternfly components, e.g.: cards & tables:
>> https://rawgit.com/patternfly/patternfly/master/tests/cards.html
>>
>> For the heatmap I used http://cal-heatmap.com/#start which is based on
>> d3js.
>> There is also an angularjs directive that could be used as well.
>> https://github.com/shekhargulati/angular-cal-heatmap-directive
>>
>> The current hacky code can be found here.:
>>
>> https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dash...
>>
>> The relevant commit is:
>>
>>
https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2dd...
>>
>> The code still needs a decent amount of polishing but I wanted to share
>> this with
>> you guys first to see if this could make it into Keycloak at some point.
>>
>> Cheers,
>> Thomas
>>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
>
>
4:27 a.m.
Answers inline...
2016-05-30 10:56 GMT+02:00 Stian Thorgersen <sthorger(a)redhat.com>:
One worry here is that this is pretty much just a rip off from the
Auth0
dashboard. That's not ideal IMO and we should have our own designed, maybe
we should include UXP guys from Red Hat to do that. If we list the details
that we want to display they can then design one for us.
Yes I took some inspirations from Auth0 and I fully agree - as said this
was a just a PoC as a base for discussion ;-)
I'm sure your UX guys are better at frontend stuff than I am :)
On 30 May 2016 at 09:51, Thomas Darimont <thomas.darimont(a)googlemail.com>
wrote:
> * What's the (+) under 7 total users?
>
> -> currently it's only a placeholder but the idea is to link to the
> "create new user" page
>
That wasn't clear to me at least. Not sure we need a link to create user
from the dashboard.
That came with the patternfly "card" templated I used as a base and
thought that might be useful (like a set of quick common operations).
>
> * What's the purpose of "Logins along the year" - it looks cool, but
I'm
> not sure how it'd be used.
>
> It would also require storing events for the whole year.
>
> -> This heatmap gives you an idea of the overall realm usage over the
> year at a glance - which allows
>
> to recognize patterns visually if the thresholds are calibrated
> accordingly.
>
> It could also be used to identify login problems e.g. after rolling out a
> new version - fewer logins than before
>
> could indicate problems for some users.
>
Wouldn't a simple graph do the same? In either case you could only display
as much data as available in the database, so it would be good if only
available months are displayed.
I think this is just a matter of specifying the "end" date for the
cal-heatmap.
Note that the cal-heatmap widget also supports browsing "past" login
activity which might be quite handy at times. Having the calendar shown
for the whole year leads to a more stable UI IMHO and would be there
anyways once enough login data is accumulated.
>
> * I'm not keen on having db specific views. We already support quite a
> few dbs so maintaining these would be painful.
>
> Would be better to delegate this to Hibernate if possible and use ejq
> queries.
>
> -> One could of course replace the fews with simple more simple and
> generic queries that could also
>
> be expressed via jpaql but this would require some processing within the
> keycloak server.
>
> At least for this prototype I wanted to keep the amount of code small.
>
> On the other hand view definitions for each database allow for optimal
> performance if you
>
> need to compute statistics / summaries after the fact from events.
>
> One could also compute the statistics eagerly e.g. update with each login
> (via custom EventListener).
>
> An alternative approach for computing summary / login statistics would be
> to use some kind of approximation mechanism.
>
> E.g. instead of computing the summary from the events one could also use
> a sketching data structure like a count min-sketch
>
> that is updated with each login (via custom EventListener) with an
> appropriately configured accuracy (e.g. 99%) that work with
>
> a fixed amount of memory.
>
A simple implementation based on events is probably good enough at least
initially. I appreciate the fact that it may be possible to tweak views for
each database, but we don't have the resources to do that. We already
support PostgreSQL, MySQL, MariaDB, Sybase, MSSQL, ..... For someone to
write specific views for each database is time-consuming and it also needs
to be maintained.
Having to persist additional things on top of events is also not ideal.
Sounds like an approximation or an alternative listener would have to do
that.
IMO performance of logins are much more important than the performance of
the dashboard.
As said if one would (asynchronously) process login events to update /
store the statistics / summary
directly you wouldn't need to store too much...
also the login code path to remain fast. For the daily aggregated login
information one could have a table like:
date; realmId; totalLoginCount;
totalLoginFailedCount; totalLoginBlockedCount; totalRegisterCount;
totalAbortedRegistrationsCount....
which would be maintained asynchronously - this would only require 1 new
table and an
internal (asynchronous) event processor. An "analytic events processor"
that emits
"increment opertations" which are reduced to a sum would also scale to
support
scenarios with multiple cluster nodes.
>
> * Logins/Registrations should display date and time. At least if date is
> not today.
>
> -> Date is displayed if date is not the same day ;-)
>
> Some additional remarks:
>
> -> The lower line of the cards in the upper area are currently mocked.
>
> In the login card the "red 4 | 1" is meant to indicate 4 failed logins
> and 1 login
>
> that lead to a blocked account.
>
> The "red 1" below the registrations card should indicate 1 failed
> registration
>
> attempt (e.g. something wrong in the server side). I can also imagine an
> indicator for "aborted" registration attempts.
>
That wasn't clear to me. Failed logins was clear, but not the blocked
account. You could actually argue that blocked accounts should be red as
they are "worse" than a single failed login.
You're right I think the same - didn't pay attention here ;-)
>
> -> The "latest logins" as well the "New registrations" should
actually be
> right next to each other instead of below each other.
>
> -> The REST endpoint, currently called DashboardResource, could also be
> exposed
>
> in a more generic form like "AnalyticsResource" which could then offer
>
> various time series and summaries as JSON output for consumption by
> other tools like nagios.
>
>
>
> 2016-05-30 7:47 GMT+02:00 Stian Thorgersen <sthorger(a)redhat.com>:
>
>> That's really cool and would be great to have this added to Keycloak.
>>
>> Some questions/comments:
>>
>> * What's the (+) under 7 total users?
>> * What's the purpose of "Logins along the year" - it looks cool,
but I'm
>> not sure how it'd be used. It would also require storing events for the
>> whole year.
>> * I'm not keen on having db specific views. We already support quite a
>> few dbs so maintaining these would be painful. Would be better to delegate
>> this to Hibernate if possible and use ejq queries.
>> * Logins/Registrations should display date and time. At least if date is
>> not today.
>>
>> On 29 May 2016 at 22:52, Thomas Darimont <thomas.darimont(a)googlemail.com
>> > wrote:
>>
>>> Hello group,
>>>
>>> a few months ago I raised the feature request "Activity dashboard"
in
>>> the Keycloak JIRA.
>>> https://issues.jboss.org/browse/KEYCLOAK-1840
>>>
>>> This weekend I gave this a spin and I think I got pretty far with it,
>>> see attached annotated screenshot.
>>>
>>> The idea was to leverage the information from the stored event data
>>> to compute some Keycloak usage statistics over time.
>>> My current prototype supports JPA (user / event) storage provider
>>> and works with postgresql but could be adapted to other databases
>>> including MongoDB.
>>>
>>> Since I need to compute the usage statistics based on the event data,
>>> events need to be stored and some views (3) need to be defined to
>>> make the data accessible from JPA in a generic fashion.
>>>
>>> Since the queries are quite complex I wanted to keep them out
>>> of the code and therefore used named native queries via orm.xml.
>>> The actual queries use some database specific date/time functions
>>> that I wanted to keep out of the code - thus I created views
>>> that could be adapted for each database and provisioned via liquibase.
>>>
>>> The view definitions can be found here:
>>> https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e
>>>
>>> For MongoDB one could define appropriate aggregation framework pipelines
>>> to express the same query logic.
>>>
>>> I basically exposed the data from those views per realm via a newly
>>> introduced AnalyticsProvider interface that is accessible via
>>> KeycloakSession.
>>>
>>> Data from this AnalyticsProvider is then exposed as a REST resource
>>> called "DashboardResource".
>>> Data from this REST endpoint is then consumed by the admin frontend in
>>> a new section
>>> called "dashboard".
>>>
>>> In the frontend I used basic patternfly components, e.g.: cards &
>>> tables:
>>> https://rawgit.com/patternfly/patternfly/master/tests/cards.html
>>>
>>> For the heatmap I used http://cal-heatmap.com/#start which is based on
>>> d3js.
>>> There is also an angularjs directive that could be used as well.
>>> https://github.com/shekhargulati/angular-cal-heatmap-directive
>>>
>>> The current hacky code can be found here.:
>>>
>>>
https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dash...
>>>
>>> The relevant commit is:
>>>
>>>
https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2dd...
>>>
>>> The code still needs a decent amount of polishing but I wanted to share
>>> this with
>>> you guys first to see if this could make it into Keycloak at some point.
>>>
>>> Cheers,
>>> Thomas
>>>
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>>
>>
>
11:15 p.m.
I didn't read whole thread on this: Having a JPA event store would be
bad performance? Isn't there more than one even per login? That means
multiple DB inserts per login just to gather stats. Stats that would be
looked at rarely (once a day? once a week? once a month?) Just
something to think about.
On 5/29/16 4:52 PM, Thomas Darimont wrote:
Hello group,
a few months ago I raised the feature request "Activity dashboard" in
the Keycloak JIRA.
https://issues.jboss.org/browse/KEYCLOAK-1840
This weekend I gave this a spin and I think I got pretty far with it,
see attached annotated screenshot.
The idea was to leverage the information from the stored event data
to compute some Keycloak usage statistics over time.
My current prototype supports JPA (user / event) storage provider
and works with postgresql but could be adapted to other databases
including MongoDB.
Since I need to compute the usage statistics based on the event data,
events need to be stored and some views (3) need to be defined to
make the data accessible from JPA in a generic fashion.
Since the queries are quite complex I wanted to keep them out
of the code and therefore used named native queries via orm.xml.
The actual queries use some database specific date/time functions
that I wanted to keep out of the code - thus I created views
that could be adapted for each database and provisioned via liquibase.
The view definitions can be found here:
https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e
For MongoDB one could define appropriate aggregation framework pipelines
to express the same query logic.
I basically exposed the data from those views per realm via a newly
introduced AnalyticsProvider interface that is accessible via
KeycloakSession.
Data from this AnalyticsProvider is then exposed as a REST resource
called "DashboardResource".
Data from this REST endpoint is then consumed by the admin frontend in
a new section
called "dashboard".
In the frontend I used basic patternfly components, e.g.: cards & tables:
https://rawgit.com/patternfly/patternfly/master/tests/cards.html
For the heatmap I used http://cal-heatmap.com/#start which is based on
d3js.
There is also an angularjs directive that could be used as well.
https://github.com/shekhargulati/angular-cal-heatmap-directive
The current hacky code can be found here.:
https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dash...
The relevant commit is:
https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2dd...
The code still needs a decent amount of polishing but I wanted to
share this with
you guys first to see if this could make it into Keycloak at some point.
Cheers,
Thomas
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
12:42 p.m.
Hello Bill,
I didn't say that using JPA to store event data would lead to bad
performance :-)
it's just that querying and aggregating event data via JPA(QL) is a pain
...
and I didn't want to potentially pull 100k+ events into memory to compute
the stats in Java.
I wanted to keep the processing in the Keycloak JVM as low as possible to
not risk any
OOME scenarios (though one could also use batch fetching here...).
That's the reason why I used a view that uses database specific means to do
so.
With that said recording events with JPA is fine IMHO - though it could be
done asynchronously in the background.
My current approach for computing stats only relies on the already stored
event_entity data,
which of course needs to be persisted then - currently I don't need to
store anything.
But since the event_entity data can be deleted I'd proposed to periodically
compute
aggregates based on those when a day is over. It could then compute stuff
like:
* totalUserCount - how many users do we have in the realm?
* totalLoginCount - how many users did login on that
particular day?
* totalLoginFailedCount - how many logins failed on that day?
* totalLoginBlockedCount - how many failed logins lead to blocked
accounts?
* totalRegisterCount - how many users did register on that day?
* totalAbortedRegistrationsCount - how many users abandonned the
registration?
- per realm per day.
Similar stats could also be held by each keycloak instance in-memory and
shared via infinispan.
Those stats could then also periodically aggregated (every 5 mins) to give
information near real-time.
Regarding the rare usage of this info I'd argue that the dashboard should
actually be the first
page that an administrator sees when logging in to Keycloak (after initial
setup of course) -
I think it gives you a lot more than just a few stats - e.g. the
possibility to login
see patterns and identify potential problems this probably consulted
multiple times a day by multiple
people - especially if one uses a lot of different tenants.
Cheers,
Thomas
2016-05-31 6:15 GMT+02:00 Bill Burke <bburke(a)redhat.com>:
I didn't read whole thread on this: Having a JPA event store
would be bad
performance? Isn't there more than one even per login? That means
multiple DB inserts per login just to gather stats. Stats that would be
looked at rarely (once a day? once a week? once a month?) Just something
to think about.
On 5/29/16 4:52 PM, Thomas Darimont wrote:
Hello group,
a few months ago I raised the feature request "Activity dashboard" in the
Keycloak JIRA.
https://issues.jboss.org/browse/KEYCLOAK-1840
This weekend I gave this a spin and I think I got pretty far with it,
see attached annotated screenshot.
The idea was to leverage the information from the stored event data
to compute some Keycloak usage statistics over time.
My current prototype supports JPA (user / event) storage provider
and works with postgresql but could be adapted to other databases
including MongoDB.
Since I need to compute the usage statistics based on the event data,
events need to be stored and some views (3) need to be defined to
make the data accessible from JPA in a generic fashion.
Since the queries are quite complex I wanted to keep them out
of the code and therefore used named native queries via orm.xml.
The actual queries use some database specific date/time functions
that I wanted to keep out of the code - thus I created views
that could be adapted for each database and provisioned via liquibase.
The view definitions can be found here:
https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e
For MongoDB one could define appropriate aggregation framework pipelines
to express the same query logic.
I basically exposed the data from those views per realm via a newly
introduced AnalyticsProvider interface that is accessible via
KeycloakSession.
Data from this AnalyticsProvider is then exposed as a REST resource called
"DashboardResource".
Data from this REST endpoint is then consumed by the admin frontend in a
new section
called "dashboard".
In the frontend I used basic patternfly components, e.g.: cards & tables:
https://rawgit.com/patternfly/patternfly/master/tests/cards.html
For the heatmap I used http://cal-heatmap.com/#start which is based on
d3js.
There is also an angularjs directive that could be used as well.
https://github.com/shekhargulati/angular-cal-heatmap-directive
The current hacky code can be found here.:
https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dash...
The relevant commit is:
https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2dd...
The code still needs a decent amount of polishing but I wanted to share
this with
you guys first to see if this could make it into Keycloak at some point.
Cheers,
Thomas
_______________________________________________
keycloak-dev mailing
listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
1:52 a.m.
On 31 May 2016 at 19:42, Thomas Darimont <thomas.darimont(a)googlemail.com>
wrote:
Hello Bill,
I didn't say that using JPA to store event data would lead to bad
performance :-)
it's just that querying and aggregating event data via JPA(QL) is a pain
...
and I didn't want to potentially pull 100k+ events into memory to compute
the stats in Java.
I wanted to keep the processing in the Keycloak JVM as low as possible to
not risk any
OOME scenarios (though one could also use batch fetching here...).
That's the reason why I used a view that uses database specific means to
do so.
With that said recording events with JPA is fine IMHO - though it could be
done asynchronously in the background.
My current approach for computing stats only relies on the already stored
event_entity data,
which of course needs to be persisted then - currently I don't need to
store anything.
But since the event_entity data can be deleted I'd proposed to
periodically compute
aggregates based on those when a day is over. It could then compute stuff
like:
* totalUserCount - how many users do we have in the realm?
* totalLoginCount - how many users did login on that
particular day?
* totalLoginFailedCount - how many logins failed on that day?
* totalLoginBlockedCount - how many failed logins lead to blocked
accounts?
* totalRegisterCount - how many users did register on that day?
* totalAbortedRegistrationsCount - how many users abandonned the
registration?
- per realm per day.
That sounds like a much better approach. Actually it could run every N
minutes with the timer provider. If it only fetches the last N minutes of
events it's not going to be a huge query, but probably still worth doing it
in batches (1K max at the time or something?).
One issue here is that the timer is currently executed on all nodes, while
this particular one should only run on one node.
Similar stats could also be held by each keycloak instance in-memory and
shared via infinispan.
Those stats could then also periodically aggregated (every 5 mins) to give
information near real-time.
Not sure that's necessary. Having the timer run periodically should be
sufficient. It should definitively be a secondary requirement and possible
to disable.
Regarding the rare usage of this info I'd argue that the dashboard should
actually be the first
page that an administrator sees when logging in to Keycloak (after initial
setup of course) -
I think it gives you a lot more than just a few stats - e.g. the
possibility to login
see patterns and identify potential problems this probably consulted
multiple times a day by multiple
people - especially if one uses a lot of different tenants.
I agree it should be the main page, but thousands of users could be
logging-in a second, while only one or two admins login to the admin
console a day. So compared to logins, it's rare.
Cheers,
Thomas
2016-05-31 6:15 GMT+02:00 Bill Burke <bburke(a)redhat.com>:
> I didn't read whole thread on this: Having a JPA event store would be
> bad performance? Isn't there more than one even per login? That means
> multiple DB inserts per login just to gather stats. Stats that would be
> looked at rarely (once a day? once a week? once a month?) Just something
> to think about.
>
> On 5/29/16 4:52 PM, Thomas Darimont wrote:
>
> Hello group,
>
> a few months ago I raised the feature request "Activity dashboard" in the
> Keycloak JIRA.
> https://issues.jboss.org/browse/KEYCLOAK-1840
>
> This weekend I gave this a spin and I think I got pretty far with it,
> see attached annotated screenshot.
>
> The idea was to leverage the information from the stored event data
> to compute some Keycloak usage statistics over time.
> My current prototype supports JPA (user / event) storage provider
> and works with postgresql but could be adapted to other databases
> including MongoDB.
>
> Since I need to compute the usage statistics based on the event data,
> events need to be stored and some views (3) need to be defined to
> make the data accessible from JPA in a generic fashion.
>
> Since the queries are quite complex I wanted to keep them out
> of the code and therefore used named native queries via orm.xml.
> The actual queries use some database specific date/time functions
> that I wanted to keep out of the code - thus I created views
> that could be adapted for each database and provisioned via liquibase.
>
> The view definitions can be found here:
> https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e
>
> For MongoDB one could define appropriate aggregation framework pipelines
> to express the same query logic.
>
> I basically exposed the data from those views per realm via a newly
> introduced AnalyticsProvider interface that is accessible via
> KeycloakSession.
>
> Data from this AnalyticsProvider is then exposed as a REST resource
> called "DashboardResource".
> Data from this REST endpoint is then consumed by the admin frontend in a
> new section
> called "dashboard".
>
> In the frontend I used basic patternfly components, e.g.: cards & tables:
> https://rawgit.com/patternfly/patternfly/master/tests/cards.html
>
> For the heatmap I used http://cal-heatmap.com/#start which is based on
> d3js.
> There is also an angularjs directive that could be used as well.
> https://github.com/shekhargulati/angular-cal-heatmap-directive
>
> The current hacky code can be found here.:
>
> https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dash...
>
> The relevant commit is:
>
>
https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2dd...
>
> The code still needs a decent amount of polishing but I wanted to share
> this with
> you guys first to see if this could make it into Keycloak at some point.
>
> Cheers,
> Thomas
>
>
> _______________________________________________
> keycloak-dev mailing
listkeycloak-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/keycloak-dev
>
>
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
3115
days inactive
3133
days old
7 comments
3 participants
participants (3)
-
Bill Burke -
Stian Thorgersen -
Thomas Darimont