<div dir="ltr">Answers inline...<br><div class="gmail_extra"><br><div class="gmail_quote">2016-05-30 10:56 GMT+02:00 Stian Thorgersen <span dir="ltr"><<a href="mailto:sthorger@redhat.com" target="_blank">sthorger@redhat.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr">One worry here is that this is pretty much just a rip off from the Auth0 dashboard. That's not ideal IMO and we should have our own designed, maybe we should include UXP guys from Red Hat to do that. If we list the details that we want to display they can then design one for us.<br></div></blockquote><div><br></div><div>Yes I took some inspirations from Auth0 and I fully agree - as said this was a just a PoC as a base for discussion ;-)</div><div>I'm sure your UX guys are better at frontend stuff than I am :)</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On 30 May 2016 at 09:51, Thomas Darimont <span dir="ltr"><<a href="mailto:thomas.darimont@googlemail.com" target="_blank">thomas.darimont@googlemail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><span><span><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">* What's the (+) under 7 total users?</span></p></span><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-> currently it's only a placeholder but the idea is to link to the "create new user" page</span></p></span></div></div></blockquote><div><br></div></span><div>That wasn't clear to me at least. Not sure we need a link to create user from the dashboard.</div></div></div></div></blockquote><div><br></div><div>That came with the patternfly "card" templated I used as a base and </div><div>thought that might be useful (like a set of quick common operations).</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><span><span><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">* What's the purpose of "Logins along the year" - it looks cool, but I'm not sure how it'd be used.</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> It would also require storing events for the whole year.</span></p></span><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-> This heatmap gives you an idea of the overall realm usage over the year at a glance - which allows </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> to recognize patterns visually if the thresholds are calibrated accordingly.</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> It could also be used to identify login problems e.g. after rolling out a new version - fewer logins than before </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> could indicate problems for some users.</span></p></span></div></div></blockquote><div><br></div></span><div>Wouldn't a simple graph do the same? In either case you could only display as much data as available in the database, so it would be good if only available months are displayed.</div></div></div></div></blockquote><div><br></div><div>I think this is just a matter of specifying the "end" date for the cal-heatmap. </div><div>Note that the cal-heatmap widget also supports browsing "past" login </div><div>activity which might be quite handy at times. Having the calendar shown </div><div>for the whole year leads to a more stable UI IMHO and would be there</div><div>anyways once enough login data is accumulated.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><span><span><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">* I'm not keen on having db specific views. We already support quite a few dbs so maintaining these would be painful. </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> Would be better to delegate this to Hibernate if possible and use ejq queries.</span></p></span><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-> One could of course replace the fews with simple more simple and generic queries that could also </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> be expressed via jpaql but this would require some processing within the keycloak server. </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> At least for this prototype I wanted to keep the amount of code small.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> On the other hand view definitions for each database allow for optimal performance if you </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> need to compute statistics / summaries after the fact from events. </span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> One could also compute the statistics eagerly e.g. update with each login (via custom EventListener).</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> An alternative approach for computing summary / login statistics would be to use some kind of approximation mechanism.</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> E.g. instead of computing the summary from the events one could also use a sketching data structure like a count min-sketch </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> that is updated with </span><span style="font-family:Arial;font-size:14.6667px;white-space:pre-wrap;line-height:1.38;background-color:transparent">each login (via custom EventListener) with an appropriately configured accuracy (e.g. 99%) that work with </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;font-size:14.6667px;white-space:pre-wrap;line-height:1.38;background-color:transparent"> a fixed amount of memory.</span></p></span></div></div></blockquote><div><br></div></span><div>A simple implementation based on events is probably good enough at least initially. I appreciate the fact that it may be possible to tweak views for each database, but we don't have the resources to do that. We already support PostgreSQL, MySQL, MariaDB, Sybase, MSSQL, ..... For someone to write specific views for each database is time-consuming and it also needs to be maintained.</div><div><br></div><div>Having to persist additional things on top of events is also not ideal. Sounds like an approximation or an alternative listener would have to do that.</div><div><br></div><div>IMO performance of logins are much more important than the performance of the dashboard.</div></div></div></div></blockquote><div><br></div><div>As said if one would (asynchronously) process login events to update / store the statistics / summary </div><div>directly you wouldn't need to store too much...</div><div>also the login code path to remain fast. For the daily aggregated login information one could have a table like:</div><div><br></div><div>date; realmId; totalLoginCount; totalLoginFailedCount; totalLoginBlockedCount; totalRegisterCount; totalAbortedRegistrationsCount....</div><div><br></div><div>which would be maintained asynchronously - this would only require 1 new table and an </div><div>internal (asynchronous) event processor. An "analytic events processor" that emits </div><div>"increment opertations" which are reduced to a sum would also scale to support </div><div>scenarios with multiple cluster nodes.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><span><span><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">* Logins/Registrations should display date and time. At least if date is not today.</span></p></span><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-> Date is displayed if date is not the same day ;-)</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">Some additional remarks:</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-> The lower line of the cards in the upper area are currently mocked.</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> In the login card the "red 4 | 1" is meant to indicate 4 failed logins and 1 login</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> that lead to a blocked account.</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> The "red 1" below the registrations card should indicate 1 failed registration </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> attempt (e.g. something wrong in the server side). I can also imagine an indicator for "aborted" registration attempts.</span></p></span></div></div></blockquote><div><br></div></span><div>That wasn't clear to me. Failed logins was clear, but not the blocked account. You could actually argue that blocked accounts should be red as they are "worse" than a single failed login.</div></div></div></div></blockquote><div>You're right I think the same - didn't pay attention here ;-)</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><span><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-> The "latest logins" as well the "New registrations" should actually be right next to each other instead of below each other.</span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent">-> The REST endpoint, currently called DashboardResource, could also be exposed</span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> in a more generic form like "AnalyticsResource" which could then offer </span></p><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"> various time series and summaries as JSON output for consumption by other tools like nagios.</span></p><div><span style="font-size:14.6667px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;background-color:transparent"><br></span></div></span></div><div><div><div><br></div><div class="gmail_extra"><br><div class="gmail_quote">2016-05-30 7:47 GMT+02:00 Stian Thorgersen <span dir="ltr"><<a href="mailto:sthorger@redhat.com" target="_blank">sthorger@redhat.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr">That's really cool and would be great to have this added to Keycloak.<div><br></div><div>Some questions/comments:</div><div><br></div><div>* What's the (+) under 7 total users?</div><div>* What's the purpose of "Logins along the year" - it looks cool, but I'm not sure how it'd be used. It would also require storing events for the whole year.</div><div>* I'm not keen on having db specific views. We already support quite a few dbs so maintaining these would be painful. Would be better to delegate this to Hibernate if possible and use ejq queries.</div><div>* Logins/Registrations should display date and time. At least if date is not today.</div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On 29 May 2016 at 22:52, Thomas Darimont <span dir="ltr"><<a href="mailto:thomas.darimont@googlemail.com" target="_blank">thomas.darimont@googlemail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div><div><div dir="ltr"><div>Hello group,</div><div><br></div><div>a few months ago I raised the feature request "Activity dashboard" in the Keycloak JIRA.</div><div><a href="https://issues.jboss.org/browse/KEYCLOAK-1840" target="_blank">https://issues.jboss.org/browse/KEYCLOAK-1840</a></div><div><br></div><div>This weekend I gave this a spin and I think I got pretty far with it,</div><div>see attached annotated screenshot.</div><div><br></div><div>The idea was to leverage the information from the stored event data</div><div>to compute some Keycloak usage statistics over time.</div><div>My current prototype supports JPA (user / event) storage provider</div><div>and works with postgresql but could be adapted to other databases including MongoDB.</div><div><br></div><div>Since I need to compute the usage statistics based on the event data,</div><div>events need to be stored and some views (3) need to be defined to</div><div>make the data accessible from JPA in a generic fashion.</div><div><br></div><div>Since the queries are quite complex I wanted to keep them out</div><div>of the code and therefore used named native queries via orm.xml.</div><div>The actual queries use some database specific date/time functions</div><div>that I wanted to keep out of the code - thus I created views </div><div>that could be adapted for each database and provisioned via liquibase.</div><div><br></div><div>The view definitions can be found here:</div><div><a href="https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e" target="_blank">https://gist.github.com/thomasdarimont/24e11be101c6ed8773f22e1defc5d66e</a></div><div><br></div><div>For MongoDB one could define appropriate aggregation framework pipelines</div><div>to express the same query logic.</div><div><br></div><div>I basically exposed the data from those views per realm via a newly </div><div>introduced AnalyticsProvider interface that is accessible via KeycloakSession.</div><div><br></div><div>Data from this AnalyticsProvider is then exposed as a REST resource called "DashboardResource".</div><div>Data from this REST endpoint is then consumed by the admin frontend in a new section</div><div>called "dashboard".</div><div><br></div><div>In the frontend I used basic patternfly components, e.g.: cards & tables:</div><div><a href="https://rawgit.com/patternfly/patternfly/master/tests/cards.html" target="_blank">https://rawgit.com/patternfly/patternfly/master/tests/cards.html</a></div><div><br></div><div>For the heatmap I used <a href="http://cal-heatmap.com/#start" target="_blank">http://cal-heatmap.com/#start</a> which is based on d3js.</div><div>There is also an angularjs directive that could be used as well.</div><div><a href="https://github.com/shekhargulati/angular-cal-heatmap-directive" target="_blank">https://github.com/shekhargulati/angular-cal-heatmap-directive</a></div><div><br></div><div>The current hacky code can be found here.:</div><div><a href="https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dashboard" target="_blank">https://github.com/thomasdarimont/keycloak/commits/poc/KEYCLOAK-1840-dashboard</a></div><div><br></div><div>The relevant commit is:</div><div><a href="https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2ddbaf27961f2a852203" target="_blank">https://github.com/thomasdarimont/keycloak/commit/40a7956f8e547edc148d2ddbaf27961f2a852203</a></div><div><br></div><div>The code still needs a decent amount of polishing but I wanted to share this with</div><div>you guys first to see if this could make it into Keycloak at some point.</div><div><br></div><div>Cheers,</div><div>Thomas</div></div>
<br></div></div>_______________________________________________<br>
keycloak-dev mailing list<br>
<a href="mailto:keycloak-dev@lists.jboss.org" target="_blank">keycloak-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/keycloak-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/keycloak-dev</a><br></blockquote></div><br></div>
</blockquote></div><br></div></div></div></div>
</blockquote></div></div></div><br></div></div>
</blockquote></div><br></div></div>