[keycloak-dev] Operational monitoring of Keycloak server
Vlastimil Elias
velias at redhat.com
Thu Jul 16 02:15:29 EDT 2015
Hi,
OK, if DB is "realm" and "users" then it have to be well documented as
it is not obvious for people without deeper knowledge of KC.
Do we have JIRA issue for this or should I create one?
Vl.
On 14.7.2015 10:06, Stian Thorgersen wrote:
>
> ----- Original Message -----
>> From: "Vlastimil Elias" <velias at redhat.com>
>> To: "Stian Thorgersen" <stian at redhat.com>
>> Cc: "Scott Rossillo" <srossillo at smartling.com>, keycloak-dev at lists.jboss.org
>> Sent: Tuesday, 14 July, 2015 9:58:36 AM
>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>
>> On 14.7.2015 09:14, Stian Thorgersen wrote:
>>> ----- Original Message -----
>>>> From: "Vlastimil Elias" <velias at redhat.com>
>>>> To: "Stian Thorgersen" <stian at redhat.com>
>>>> Cc: "Scott Rossillo" <srossillo at smartling.com>,
>>>> keycloak-dev at lists.jboss.org
>>>> Sent: Tuesday, 14 July, 2015 8:56:43 AM
>>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>>>
>>>>
>>>>
>>>> On 14.7.2015 08:42, Stian Thorgersen wrote:
>>>>> Ok so we add one that just returns OK or ERROR which doesn't require
>>>>> authentication, but then we add another that can return more details,
>>>>> including stats and what the error is.
>>>> Yep. Simple http endpoinds for now, later all this info may be expose
>>>> over DMR or some other mechanism if necessary.
>>>>> For full health we should check all external services that are used (db,
>>>>> identity providers and user federation providers). Cluster sync will
>>>>> depend on what Infinispan exposes, but I imagine they'll have the
>>>>> required
>>>>> details avail.
>>>> Beside endpoint for full health check (which should summarize all
>>>> services into one flag if I understand correctly, which is good for
>>>> loadbalancer) we should expose separate endpoint for each service
>>>> health, as it is better for operational monitoring systems like Nagios.
>>>> Admins are better informed what is going wrong and they can resolve
>>>> situation more quickly without necessity to search cause of general
>>>> failure flag somewhere else.
>>> We could have a single health endpoint that returns a 200 or 500 and also
>>> returns a json doc with details. Something like:
>>>
>>> {
>>> realm: ok,
>>> users: ok,
>>> realmCache: error,
>>> userCache: error,
>>> identity-providers: {
>>> google: ok,
>>> facebook: error
>>> },
>>> user-federation: {
>>> ldap: ok
>>> }
>>> }
>>>
>>> I wonder if that's to much details to expose without authentication.
>> This should work, but I miss info about DB connection here. It is
>> crucial from operational point of view as this allows sysadmins quickly
>> detect that connection to DB is broken.
>>
>> I believe this kind of info can be safely exposed without
>> authentication. If the endpoint is clearly documented then access can be
>> always restricted on some network infra (webserver/proxy) ahead of KC.
> DB is "realm" and "users".
>
>>>> Vl.
>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Vlastimil Elias" <velias at redhat.com>
>>>>>> To: "Scott Rossillo" <srossillo at smartling.com>, "Stian Thorgersen"
>>>>>> <stian at redhat.com>
>>>>>> Cc: keycloak-dev at lists.jboss.org
>>>>>> Sent: Tuesday, 14 July, 2015 8:34:29 AM
>>>>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>>>>>
>>>>>> Yap, beside server-stat endpoint Stian talked about we should add some
>>>>>> server-health endpoint also to be used for remote health monitoring by
>>>>>> load balancers or systems like Nagios.
>>>>>> Simple http endpoint without authentication (as it doesn't leak any
>>>>>> sensitive information) is typically simplest way how to do this
>>>>>> monitoring.
>>>>>>
>>>>>> Vl.
>>>>>>
>>>>>> On 14.7.2015 05:13, Scott Rossillo wrote:
>>>>>>> Some type of health page would be great too for load balancers to
>>>>>>> monitor. Something that doesn't leak internal information but checks
>>>>>>> behind the scenes that:
>>>>>>> 1. Server can reach its databas(es)
>>>>>>> 2. Server cluster sync is working
>>>>>>> 3. Server can reach federation providers, etc.
>>>>>>> Endpoint should respond to get requests and return an http status
>>>>>>> reflective of server state.
>>>>>>>
>>>>>>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen <stian at redhat.com
>>>>>>> <mailto:stian at redhat.com>> wrote:
>>>>>>>
>>>>>>> So looks like we're at agreement to add the additional info you
>>>>>>> wanted to server info page.
>>>>>>>
>>>>>>> How about we add an additional endpoint server-stat that can
>>>>>>> collect some stats about the server?
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>> > From: "Vlastimil Elias" <velias at redhat.com
>>>>>>> <mailto:velias at redhat.com>>
>>>>>>> > To: "Stian Thorgersen" <stian at redhat.com
>>>>>>> > <mailto:stian at redhat.com>>
>>>>>>> > Cc: keycloak-dev at lists.jboss.org
>>>>>>> <mailto:keycloak-dev at lists.jboss.org>
>>>>>>> > Sent: Monday, 13 July, 2015 5:06:34 PM
>>>>>>> > Subject: Re: [keycloak-dev] Operational monitoring of Keycloak
>>>>>>> server
>>>>>>> >
>>>>>>> > Looks like I have to look at WildFly/EAP DMR to see what is
>>>>>>> possible to
>>>>>>> > do with it, as I'm not sure if it is about remote monitoring
>>>>>>> also and
>>>>>>> > if/how it can be use from monitoring systems like Splunk.
>>>>>>> >
>>>>>>> > Vl.
>>>>>>> >
>>>>>>> > On 13.7.2015 15:26, Stian Thorgersen wrote:
>>>>>>> > > In WildFly/EAP that's DMR right? We're planning to make
>>>>>>> Keycloak managable
>>>>>>> > > through that as well. For example everything that goes into
>>>>>>> > > keycloak-server.json will eventually be moved to
>>>>>>> standalone.xml. Same with
>>>>>>> > > admin endpoints, everything you can do there you'll
>>>>>>> > > eventually
>>>>>>> be able to
>>>>>>> > > do through DMR and jboss-cli as well.
>>>>>>> > >
>>>>>>> > > However, IMO it would make sense to at least expose Keycloak
>>>>>>> specific
>>>>>>> > > information through the admin endpoints and console as well.
>>>>>>> Such number
>>>>>>> > > of sessions, etc..
>>>>>>> > >
>>>>>>> > > ----- Original Message -----
>>>>>>> > >> From: "Vlastimil Elias" <velias at redhat.com
>>>>>>> <mailto:velias at redhat.com>>
>>>>>>> > >> To: keycloak-dev at lists.jboss.org
>>>>>>> <mailto:keycloak-dev at lists.jboss.org>
>>>>>>> > >> Sent: Monday, 13 July, 2015 3:17:16 PM
>>>>>>> > >> Subject: [keycloak-dev] Operational monitoring of Keycloak
>>>>>>> > >> server
>>>>>>> > >>
>>>>>>> > >> Hi,
>>>>>>> > >>
>>>>>>> > >> as we deployed KC to production mode for
>>>>>>> https://developers.redhat.com
>>>>>>> > >> we started to think about operational monitoring, for
>>>>>>> > >> example
>>>>>>> from
>>>>>>> > >> Nagios or other systems of this type.
>>>>>>> > >>
>>>>>>> > >> KC user guide doesn't contain any chapter covering this
>>>>>>> topic, also no
>>>>>>> > >> any success over google search, so looks like KC doesn't
>>>>>>> > >> have
>>>>>>> > >> any
>>>>>>> > >> solution for this yet.
>>>>>>> > >> But I believe this is an important area which must be solved
>>>>>>> when KC is
>>>>>>> > >> used for production.
>>>>>>> > >>
>>>>>>> > >> I can imagine monitoring of JDBC connection if JPA is used,
>>>>>>> monitoring
>>>>>>> > >> of Mongo connection if used as store, monitoring of LDAP
>>>>>>> connection if
>>>>>>> > >> LDAP federation is used etc.
>>>>>>> > >> Also some statistics like numbers of active sso session,
>>>>>>> number of
>>>>>>> > >> logins per minute etc should be provided there.
>>>>>>> > >>
>>>>>>> > >> Monitoring is not about Keycloak core itself, it should be
>>>>>>> available for
>>>>>>> > >> extension developers also. For example we implemented own
>>>>>>> > >> UserFederationProvider which calls backend REST services.
>>>>>>> > >> We should be able to add info about this integration into
>>>>>>> monitoring
>>>>>>> > >> endpoint to be able to catch problems with this REST API.
>>>>>>> > >>
>>>>>>> > >> It should be probably implemented same way as used by
>>>>>>> > >> underlying
>>>>>>> > >> WildFly/EAP (JPA/JDBC is probably available for monitoring
>>>>>>> there). I'm
>>>>>>> > >> not sure if JMX is used there still or if some new framework
>>>>>>> > >> is
>>>>>>> > >> available for it.
>>>>>>> > >> Or KC should use some form of KC REST API for this, which
>>>>>>> should be
>>>>>>> > >> extended by additional info from KC extensions?
>>>>>>> > >>
>>>>>>> > >> What do you think?
>>>>>>> > >>
>>>>>>> > >> Vlastimil
>>>>>>> > >>
>>>>>>> > >> P.S we have https://issues.jboss.org/browse/RHD-552 for Red
>>>>>>> > >> Hat
>>>>>>> > >> Developer instance of KC
>>>>>>> > >>
>>>>>>> > >> --
>>>>>>> > >> Vlastimil Elias
>>>>>>> > >> Principal Software Engineer
>>>>>>> > >> jboss.org <http://jboss.org> Development Team
>>>>>>> > >>
>>>>>>> > >> _______________________________________________
>>>>>>> > >> keycloak-dev mailing list
>>>>>>> > >> keycloak-dev at lists.jboss.org
>>>>>>> <mailto:keycloak-dev at lists.jboss.org>
>>>>>>> > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>> > >>
>>>>>>> >
>>>>>>> > --
>>>>>>> > Vlastimil Elias
>>>>>>> > Principal Software Engineer
>>>>>>> > jboss.org <http://jboss.org> Development Team
>>>>>>> >
>>>>>>> >
>>>>>>> _______________________________________________
>>>>>>> keycloak-dev mailing list
>>>>>>> keycloak-dev at lists.jboss.org
>>>>>>> <mailto:keycloak-dev at lists.jboss.org>
>>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>
>>>>>> --
>>>>>> Vlastimil Elias
>>>>>> Principal Software Engineer
>>>>>> jboss.org Development Team
>>>>>>
>>>>>>
>>>> --
>>>> Vlastimil Elias
>>>> Principal Software Engineer
>>>> jboss.org Development Team
>>>>
>>>>
>> --
>> Vlastimil Elias
>> Principal Software Engineer
>> jboss.org Development Team
>>
>>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
More information about the keycloak-dev
mailing list