Re: [keycloak-dev] Operational monitoring of Keycloak server

From: "Vlastimil Elias" <velias(a)redhat.com&gt; To: "Stian Thorgersen" <stian(a)redhat.com&gt; Cc: "Scott Rossillo" <srossillo(a)smartling.com&gt;, keycloak-dev(a)lists.jboss.org Sent: Tuesday, 14 July, 2015 8:56:43 AM Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server On 14.7.2015 08:42, Stian Thorgersen wrote: > Ok so we add one that just returns OK or ERROR which doesn't require > authentication, but then we add another that can return more details, > including stats and what the error is. Yep. Simple http endpoinds for now, later all this info may be expose over DMR or some other mechanism if necessary. > > For full health we should check all external services that are used (db, > identity providers and user federation providers). Cluster sync will > depend on what Infinispan exposes, but I imagine they'll have the required > details avail. Beside endpoint for full health check (which should summarize all services into one flag if I understand correctly, which is good for loadbalancer) we should expose separate endpoint for each service health, as it is better for operational monitoring systems like Nagios. Admins are better informed what is going wrong and they can resolve situation more quickly without necessity to search cause of general failure flag somewhere else.

Vl. > > ----- Original Message ----- >> From: "Vlastimil Elias" <velias(a)redhat.com&gt; >> To: "Scott Rossillo" <srossillo(a)smartling.com&gt;, "Stian Thorgersen" >> <stian(a)redhat.com&gt; >> Cc: keycloak-dev(a)lists.jboss.org >> Sent: Tuesday, 14 July, 2015 8:34:29 AM >> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server >> >> Yap, beside server-stat endpoint Stian talked about we should add some >> server-health endpoint also to be used for remote health monitoring by >> load balancers or systems like Nagios. >> Simple http endpoint without authentication (as it doesn't leak any >> sensitive information) is typically simplest way how to do this >> monitoring. >> >> Vl. >> >> On 14.7.2015 05:13, Scott Rossillo wrote: >>> Some type of health page would be great too for load balancers to >>> monitor. Something that doesn't leak internal information but checks >>> behind the scenes that: >>> 1. Server can reach its databas(es) >>> 2. Server cluster sync is working >>> 3. Server can reach federation providers, etc. >>> Endpoint should respond to get requests and return an http status >>> reflective of server state. >>> >>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen <stian(a)redhat.com >>> <mailto:stian@redhat.com>> wrote: >>> >>> So looks like we're at agreement to add the additional info you >>> wanted to server info page. >>> >>> How about we add an additional endpoint server-stat that can >>> collect some stats about the server? >>> >>> ----- Original Message ----- >>> > From: "Vlastimil Elias" <velias(a)redhat.com >>> <mailto:velias@redhat.com>> >>> > To: "Stian Thorgersen" <stian(a)redhat.com >>> > <mailto:stian@redhat.com>> >>> > Cc: keycloak-dev(a)lists.jboss.org >>> <mailto:keycloak-dev@lists.jboss.org> >>> > Sent: Monday, 13 July, 2015 5:06:34 PM >>> > Subject: Re: [keycloak-dev] Operational monitoring of Keycloak >>> server >>> > >>> > Looks like I have to look at WildFly/EAP DMR to see what is >>> possible to >>> > do with it, as I'm not sure if it is about remote monitoring >>> also and >>> > if/how it can be use from monitoring systems like Splunk. >>> > >>> > Vl. >>> > >>> > On 13.7.2015 15:26, Stian Thorgersen wrote: >>> > > In WildFly/EAP that's DMR right? We're planning to make >>> Keycloak managable >>> > > through that as well. For example everything that goes into >>> > > keycloak-server.json will eventually be moved to >>> standalone.xml. Same with >>> > > admin endpoints, everything you can do there you'll eventually >>> be able to >>> > > do through DMR and jboss-cli as well. >>> > > >>> > > However, IMO it would make sense to at least expose Keycloak >>> specific >>> > > information through the admin endpoints and console as well. >>> Such number >>> > > of sessions, etc.. >>> > > >>> > > ----- Original Message ----- >>> > >> From: "Vlastimil Elias" <velias(a)redhat.com >>> <mailto:velias@redhat.com>> >>> > >> To: keycloak-dev(a)lists.jboss.org >>> <mailto:keycloak-dev@lists.jboss.org> >>> > >> Sent: Monday, 13 July, 2015 3:17:16 PM >>> > >> Subject: [keycloak-dev] Operational monitoring of Keycloak >>> > >> server >>> > >> >>> > >> Hi, >>> > >> >>> > >> as we deployed KC to production mode for >>> https://developers.redhat.com >>> > >> we started to think about operational monitoring, for example >>> from >>> > >> Nagios or other systems of this type. >>> > >> >>> > >> KC user guide doesn't contain any chapter covering this >>> topic, also no >>> > >> any success over google search, so looks like KC doesn't have >>> > >> any >>> > >> solution for this yet. >>> > >> But I believe this is an important area which must be solved >>> when KC is >>> > >> used for production. >>> > >> >>> > >> I can imagine monitoring of JDBC connection if JPA is used, >>> monitoring >>> > >> of Mongo connection if used as store, monitoring of LDAP >>> connection if >>> > >> LDAP federation is used etc. >>> > >> Also some statistics like numbers of active sso session, >>> number of >>> > >> logins per minute etc should be provided there. >>> > >> >>> > >> Monitoring is not about Keycloak core itself, it should be >>> available for >>> > >> extension developers also. For example we implemented own >>> > >> UserFederationProvider which calls backend REST services. >>> > >> We should be able to add info about this integration into >>> monitoring >>> > >> endpoint to be able to catch problems with this REST API. >>> > >> >>> > >> It should be probably implemented same way as used by >>> > >> underlying >>> > >> WildFly/EAP (JPA/JDBC is probably available for monitoring >>> there). I'm >>> > >> not sure if JMX is used there still or if some new framework is >>> > >> available for it. >>> > >> Or KC should use some form of KC REST API for this, which >>> should be >>> > >> extended by additional info from KC extensions? >>> > >> >>> > >> What do you think? >>> > >> >>> > >> Vlastimil >>> > >> >>> > >> P.S we have https://issues.jboss.org/browse/RHD-552 for Red Hat >>> > >> Developer instance of KC >>> > >> >>> > >> -- >>> > >> Vlastimil Elias >>> > >> Principal Software Engineer >>> > >> jboss.org <http://jboss.org> Development Team >>> > >> >>> > >> _______________________________________________ >>> > >> keycloak-dev mailing list >>> > >> keycloak-dev(a)lists.jboss.org >>> <mailto:keycloak-dev@lists.jboss.org> >>> > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> > >> >>> > >>> > -- >>> > Vlastimil Elias >>> > Principal Software Engineer >>> > jboss.org <http://jboss.org> Development Team >>> > >>> > >>> _______________________________________________ >>> keycloak-dev mailing list >>> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org> >>> https://lists.jboss.org/mailman/listinfo/keycloak-dev >>> >> -- >> Vlastimil Elias >> Principal Software Engineer >> jboss.org Development Team >> >> -- Vlastimil Elias Principal Software Engineer jboss.org Development Team