[keycloak-dev] Operational monitoring of Keycloak server
Vlastimil Elias
velias at redhat.com
Tue Jul 14 03:58:36 EDT 2015
On 14.7.2015 09:14, Stian Thorgersen wrote:
>
> ----- Original Message -----
>> From: "Vlastimil Elias" <velias at redhat.com>
>> To: "Stian Thorgersen" <stian at redhat.com>
>> Cc: "Scott Rossillo" <srossillo at smartling.com>, keycloak-dev at lists.jboss.org
>> Sent: Tuesday, 14 July, 2015 8:56:43 AM
>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>
>>
>>
>> On 14.7.2015 08:42, Stian Thorgersen wrote:
>>> Ok so we add one that just returns OK or ERROR which doesn't require
>>> authentication, but then we add another that can return more details,
>>> including stats and what the error is.
>> Yep. Simple http endpoinds for now, later all this info may be expose
>> over DMR or some other mechanism if necessary.
>>> For full health we should check all external services that are used (db,
>>> identity providers and user federation providers). Cluster sync will
>>> depend on what Infinispan exposes, but I imagine they'll have the required
>>> details avail.
>> Beside endpoint for full health check (which should summarize all
>> services into one flag if I understand correctly, which is good for
>> loadbalancer) we should expose separate endpoint for each service
>> health, as it is better for operational monitoring systems like Nagios.
>> Admins are better informed what is going wrong and they can resolve
>> situation more quickly without necessity to search cause of general
>> failure flag somewhere else.
> We could have a single health endpoint that returns a 200 or 500 and also returns a json doc with details. Something like:
>
> {
> realm: ok,
> users: ok,
> realmCache: error,
> userCache: error,
> identity-providers: {
> google: ok,
> facebook: error
> },
> user-federation: {
> ldap: ok
> }
> }
>
> I wonder if that's to much details to expose without authentication.
This should work, but I miss info about DB connection here. It is
crucial from operational point of view as this allows sysadmins quickly
detect that connection to DB is broken.
I believe this kind of info can be safely exposed without
authentication. If the endpoint is clearly documented then access can be
always restricted on some network infra (webserver/proxy) ahead of KC.
>
>> Vl.
>>
>>> ----- Original Message -----
>>>> From: "Vlastimil Elias" <velias at redhat.com>
>>>> To: "Scott Rossillo" <srossillo at smartling.com>, "Stian Thorgersen"
>>>> <stian at redhat.com>
>>>> Cc: keycloak-dev at lists.jboss.org
>>>> Sent: Tuesday, 14 July, 2015 8:34:29 AM
>>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>>>
>>>> Yap, beside server-stat endpoint Stian talked about we should add some
>>>> server-health endpoint also to be used for remote health monitoring by
>>>> load balancers or systems like Nagios.
>>>> Simple http endpoint without authentication (as it doesn't leak any
>>>> sensitive information) is typically simplest way how to do this
>>>> monitoring.
>>>>
>>>> Vl.
>>>>
>>>> On 14.7.2015 05:13, Scott Rossillo wrote:
>>>>> Some type of health page would be great too for load balancers to
>>>>> monitor. Something that doesn't leak internal information but checks
>>>>> behind the scenes that:
>>>>> 1. Server can reach its databas(es)
>>>>> 2. Server cluster sync is working
>>>>> 3. Server can reach federation providers, etc.
>>>>> Endpoint should respond to get requests and return an http status
>>>>> reflective of server state.
>>>>>
>>>>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen <stian at redhat.com
>>>>> <mailto:stian at redhat.com>> wrote:
>>>>>
>>>>> So looks like we're at agreement to add the additional info you
>>>>> wanted to server info page.
>>>>>
>>>>> How about we add an additional endpoint server-stat that can
>>>>> collect some stats about the server?
>>>>>
>>>>> ----- Original Message -----
>>>>> > From: "Vlastimil Elias" <velias at redhat.com
>>>>> <mailto:velias at redhat.com>>
>>>>> > To: "Stian Thorgersen" <stian at redhat.com
>>>>> > <mailto:stian at redhat.com>>
>>>>> > Cc: keycloak-dev at lists.jboss.org
>>>>> <mailto:keycloak-dev at lists.jboss.org>
>>>>> > Sent: Monday, 13 July, 2015 5:06:34 PM
>>>>> > Subject: Re: [keycloak-dev] Operational monitoring of Keycloak
>>>>> server
>>>>> >
>>>>> > Looks like I have to look at WildFly/EAP DMR to see what is
>>>>> possible to
>>>>> > do with it, as I'm not sure if it is about remote monitoring
>>>>> also and
>>>>> > if/how it can be use from monitoring systems like Splunk.
>>>>> >
>>>>> > Vl.
>>>>> >
>>>>> > On 13.7.2015 15:26, Stian Thorgersen wrote:
>>>>> > > In WildFly/EAP that's DMR right? We're planning to make
>>>>> Keycloak managable
>>>>> > > through that as well. For example everything that goes into
>>>>> > > keycloak-server.json will eventually be moved to
>>>>> standalone.xml. Same with
>>>>> > > admin endpoints, everything you can do there you'll eventually
>>>>> be able to
>>>>> > > do through DMR and jboss-cli as well.
>>>>> > >
>>>>> > > However, IMO it would make sense to at least expose Keycloak
>>>>> specific
>>>>> > > information through the admin endpoints and console as well.
>>>>> Such number
>>>>> > > of sessions, etc..
>>>>> > >
>>>>> > > ----- Original Message -----
>>>>> > >> From: "Vlastimil Elias" <velias at redhat.com
>>>>> <mailto:velias at redhat.com>>
>>>>> > >> To: keycloak-dev at lists.jboss.org
>>>>> <mailto:keycloak-dev at lists.jboss.org>
>>>>> > >> Sent: Monday, 13 July, 2015 3:17:16 PM
>>>>> > >> Subject: [keycloak-dev] Operational monitoring of Keycloak
>>>>> > >> server
>>>>> > >>
>>>>> > >> Hi,
>>>>> > >>
>>>>> > >> as we deployed KC to production mode for
>>>>> https://developers.redhat.com
>>>>> > >> we started to think about operational monitoring, for example
>>>>> from
>>>>> > >> Nagios or other systems of this type.
>>>>> > >>
>>>>> > >> KC user guide doesn't contain any chapter covering this
>>>>> topic, also no
>>>>> > >> any success over google search, so looks like KC doesn't have
>>>>> > >> any
>>>>> > >> solution for this yet.
>>>>> > >> But I believe this is an important area which must be solved
>>>>> when KC is
>>>>> > >> used for production.
>>>>> > >>
>>>>> > >> I can imagine monitoring of JDBC connection if JPA is used,
>>>>> monitoring
>>>>> > >> of Mongo connection if used as store, monitoring of LDAP
>>>>> connection if
>>>>> > >> LDAP federation is used etc.
>>>>> > >> Also some statistics like numbers of active sso session,
>>>>> number of
>>>>> > >> logins per minute etc should be provided there.
>>>>> > >>
>>>>> > >> Monitoring is not about Keycloak core itself, it should be
>>>>> available for
>>>>> > >> extension developers also. For example we implemented own
>>>>> > >> UserFederationProvider which calls backend REST services.
>>>>> > >> We should be able to add info about this integration into
>>>>> monitoring
>>>>> > >> endpoint to be able to catch problems with this REST API.
>>>>> > >>
>>>>> > >> It should be probably implemented same way as used by
>>>>> > >> underlying
>>>>> > >> WildFly/EAP (JPA/JDBC is probably available for monitoring
>>>>> there). I'm
>>>>> > >> not sure if JMX is used there still or if some new framework is
>>>>> > >> available for it.
>>>>> > >> Or KC should use some form of KC REST API for this, which
>>>>> should be
>>>>> > >> extended by additional info from KC extensions?
>>>>> > >>
>>>>> > >> What do you think?
>>>>> > >>
>>>>> > >> Vlastimil
>>>>> > >>
>>>>> > >> P.S we have https://issues.jboss.org/browse/RHD-552 for Red Hat
>>>>> > >> Developer instance of KC
>>>>> > >>
>>>>> > >> --
>>>>> > >> Vlastimil Elias
>>>>> > >> Principal Software Engineer
>>>>> > >> jboss.org <http://jboss.org> Development Team
>>>>> > >>
>>>>> > >> _______________________________________________
>>>>> > >> keycloak-dev mailing list
>>>>> > >> keycloak-dev at lists.jboss.org
>>>>> <mailto:keycloak-dev at lists.jboss.org>
>>>>> > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>> > >>
>>>>> >
>>>>> > --
>>>>> > Vlastimil Elias
>>>>> > Principal Software Engineer
>>>>> > jboss.org <http://jboss.org> Development Team
>>>>> >
>>>>> >
>>>>> _______________________________________________
>>>>> keycloak-dev mailing list
>>>>> keycloak-dev at lists.jboss.org <mailto:keycloak-dev at lists.jboss.org>
>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>
>>>> --
>>>> Vlastimil Elias
>>>> Principal Software Engineer
>>>> jboss.org Development Team
>>>>
>>>>
>> --
>> Vlastimil Elias
>> Principal Software Engineer
>> jboss.org Development Team
>>
>>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
More information about the keycloak-dev
mailing list