8:17 a.m.
Hi,
as we deployed KC to production mode for https://developers.redhat.com
we started to think about operational monitoring, for example from
Nagios or other systems of this type.
KC user guide doesn't contain any chapter covering this topic, also no
any success over google search, so looks like KC doesn't have any
solution for this yet.
But I believe this is an important area which must be solved when KC is
used for production.
I can imagine monitoring of JDBC connection if JPA is used, monitoring
of Mongo connection if used as store, monitoring of LDAP connection if
LDAP federation is used etc.
Also some statistics like numbers of active sso session, number of
logins per minute etc should be provided there.
Monitoring is not about Keycloak core itself, it should be available for
extension developers also. For example we implemented own
UserFederationProvider which calls backend REST services.
We should be able to add info about this integration into monitoring
endpoint to be able to catch problems with this REST API.
It should be probably implemented same way as used by underlying
WildFly/EAP (JPA/JDBC is probably available for monitoring there). I'm
not sure if JMX is used there still or if some new framework is
available for it.
Or KC should use some form of KC REST API for this, which should be
extended by additional info from KC extensions?
What do you think?
Vlastimil
P.S we have https://issues.jboss.org/browse/RHD-552 for Red Hat
Developer instance of KC
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
8:26 a.m.
In WildFly/EAP that's DMR right? We're planning to make Keycloak managable through
that as well. For example everything that goes into keycloak-server.json will eventually
be moved to standalone.xml. Same with admin endpoints, everything you can do there
you'll eventually be able to do through DMR and jboss-cli as well.
However, IMO it would make sense to at least expose Keycloak specific information through
the admin endpoints and console as well. Such number of sessions, etc..
----- Original Message -----
From: "Vlastimil Elias" <velias(a)redhat.com>
To: keycloak-dev(a)lists.jboss.org
Sent: Monday, 13 July, 2015 3:17:16 PM
Subject: [keycloak-dev] Operational monitoring of Keycloak server
Hi,
as we deployed KC to production mode for https://developers.redhat.com
we started to think about operational monitoring, for example from
Nagios or other systems of this type.
KC user guide doesn't contain any chapter covering this topic, also no
any success over google search, so looks like KC doesn't have any
solution for this yet.
But I believe this is an important area which must be solved when KC is
used for production.
I can imagine monitoring of JDBC connection if JPA is used, monitoring
of Mongo connection if used as store, monitoring of LDAP connection if
LDAP federation is used etc.
Also some statistics like numbers of active sso session, number of
logins per minute etc should be provided there.
Monitoring is not about Keycloak core itself, it should be available for
extension developers also. For example we implemented own
UserFederationProvider which calls backend REST services.
We should be able to add info about this integration into monitoring
endpoint to be able to catch problems with this REST API.
It should be probably implemented same way as used by underlying
WildFly/EAP (JPA/JDBC is probably available for monitoring there). I'm
not sure if JMX is used there still or if some new framework is
available for it.
Or KC should use some form of KC REST API for this, which should be
extended by additional info from KC extensions?
What do you think?
Vlastimil
P.S we have https://issues.jboss.org/browse/RHD-552 for Red Hat
Developer instance of KC
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
10:06 a.m.
Looks like I have to look at WildFly/EAP DMR to see what is possible to
do with it, as I'm not sure if it is about remote monitoring also and
if/how it can be use from monitoring systems like Splunk.
Vl.
On 13.7.2015 15:26, Stian Thorgersen wrote:
In WildFly/EAP that's DMR right? We're planning to make
Keycloak managable through that as well. For example everything that goes into
keycloak-server.json will eventually be moved to standalone.xml. Same with admin
endpoints, everything you can do there you'll eventually be able to do through DMR and
jboss-cli as well.
However, IMO it would make sense to at least expose Keycloak specific information through
the admin endpoints and console as well. Such number of sessions, etc..
----- Original Message -----
> From: "Vlastimil Elias" <velias(a)redhat.com>
> To: keycloak-dev(a)lists.jboss.org
> Sent: Monday, 13 July, 2015 3:17:16 PM
> Subject: [keycloak-dev] Operational monitoring of Keycloak server
>
> Hi,
>
> as we deployed KC to production mode for https://developers.redhat.com
> we started to think about operational monitoring, for example from
> Nagios or other systems of this type.
>
> KC user guide doesn't contain any chapter covering this topic, also no
> any success over google search, so looks like KC doesn't have any
> solution for this yet.
> But I believe this is an important area which must be solved when KC is
> used for production.
>
> I can imagine monitoring of JDBC connection if JPA is used, monitoring
> of Mongo connection if used as store, monitoring of LDAP connection if
> LDAP federation is used etc.
> Also some statistics like numbers of active sso session, number of
> logins per minute etc should be provided there.
>
> Monitoring is not about Keycloak core itself, it should be available for
> extension developers also. For example we implemented own
> UserFederationProvider which calls backend REST services.
> We should be able to add info about this integration into monitoring
> endpoint to be able to catch problems with this REST API.
>
> It should be probably implemented same way as used by underlying
> WildFly/EAP (JPA/JDBC is probably available for monitoring there). I'm
> not sure if JMX is used there still or if some new framework is
> available for it.
> Or KC should use some form of KC REST API for this, which should be
> extended by additional info from KC extensions?
>
> What do you think?
>
> Vlastimil
>
> P.S we have https://issues.jboss.org/browse/RHD-552 for Red Hat
> Developer instance of KC
>
> --
> Vlastimil Elias
> Principal Software Engineer
> jboss.org Development Team
>
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
10:18 a.m.
So looks like we're at agreement to add the additional info you wanted to server info
page.
How about we add an additional endpoint server-stat that can collect some stats about the
server?
----- Original Message -----
From: "Vlastimil Elias" <velias(a)redhat.com>
To: "Stian Thorgersen" <stian(a)redhat.com>
Cc: keycloak-dev(a)lists.jboss.org
Sent: Monday, 13 July, 2015 5:06:34 PM
Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
Looks like I have to look at WildFly/EAP DMR to see what is possible to
do with it, as I'm not sure if it is about remote monitoring also and
if/how it can be use from monitoring systems like Splunk.
Vl.
On 13.7.2015 15:26, Stian Thorgersen wrote:
> In WildFly/EAP that's DMR right? We're planning to make Keycloak managable
> through that as well. For example everything that goes into
> keycloak-server.json will eventually be moved to standalone.xml. Same with
> admin endpoints, everything you can do there you'll eventually be able to
> do through DMR and jboss-cli as well.
>
> However, IMO it would make sense to at least expose Keycloak specific
> information through the admin endpoints and console as well. Such number
> of sessions, etc..
>
> ----- Original Message -----
>> From: "Vlastimil Elias" <velias(a)redhat.com>
>> To: keycloak-dev(a)lists.jboss.org
>> Sent: Monday, 13 July, 2015 3:17:16 PM
>> Subject: [keycloak-dev] Operational monitoring of Keycloak server
>>
>> Hi,
>>
>> as we deployed KC to production mode for https://developers.redhat.com
>> we started to think about operational monitoring, for example from
>> Nagios or other systems of this type.
>>
>> KC user guide doesn't contain any chapter covering this topic, also no
>> any success over google search, so looks like KC doesn't have any
>> solution for this yet.
>> But I believe this is an important area which must be solved when KC is
>> used for production.
>>
>> I can imagine monitoring of JDBC connection if JPA is used, monitoring
>> of Mongo connection if used as store, monitoring of LDAP connection if
>> LDAP federation is used etc.
>> Also some statistics like numbers of active sso session, number of
>> logins per minute etc should be provided there.
>>
>> Monitoring is not about Keycloak core itself, it should be available for
>> extension developers also. For example we implemented own
>> UserFederationProvider which calls backend REST services.
>> We should be able to add info about this integration into monitoring
>> endpoint to be able to catch problems with this REST API.
>>
>> It should be probably implemented same way as used by underlying
>> WildFly/EAP (JPA/JDBC is probably available for monitoring there). I'm
>> not sure if JMX is used there still or if some new framework is
>> available for it.
>> Or KC should use some form of KC REST API for this, which should be
>> extended by additional info from KC extensions?
>>
>> What do you think?
>>
>> Vlastimil
>>
>> P.S we have https://issues.jboss.org/browse/RHD-552 for Red Hat
>> Developer instance of KC
>>
>> --
>> Vlastimil Elias
>> Principal Software Engineer
>> jboss.org Development Team
>>
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
10:13 p.m.
Some type of health page would be great too for load balancers to monitor.
Something that doesn't leak internal information but checks behind the
scenes that:
1. Server can reach its databas(es)
2. Server cluster sync is working
3. Server can reach federation providers, etc.
Endpoint should respond to get requests and return an http status
reflective of server state.
On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen <stian(a)redhat.com> wrote:
So looks like we're at agreement to add the additional info you
wanted to
server info page.
How about we add an additional endpoint server-stat that can collect some
stats about the server?
----- Original Message -----
> From: "Vlastimil Elias" <velias(a)redhat.com>
> To: "Stian Thorgersen" <stian(a)redhat.com>
> Cc: keycloak-dev(a)lists.jboss.org
> Sent: Monday, 13 July, 2015 5:06:34 PM
> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>
> Looks like I have to look at WildFly/EAP DMR to see what is possible to
> do with it, as I'm not sure if it is about remote monitoring also and
> if/how it can be use from monitoring systems like Splunk.
>
> Vl.
>
> On 13.7.2015 15:26, Stian Thorgersen wrote:
> > In WildFly/EAP that's DMR right? We're planning to make Keycloak
managable
> > through that as well. For example everything that goes into
> > keycloak-server.json will eventually be moved to standalone.xml. Same
with
> > admin endpoints, everything you can do there you'll eventually be able
to
> > do through DMR and jboss-cli as well.
> >
> > However, IMO it would make sense to at least expose Keycloak specific
> > information through the admin endpoints and console as well. Such
number
> > of sessions, etc..
> >
> > ----- Original Message -----
> >> From: "Vlastimil Elias" <velias(a)redhat.com>
> >> To: keycloak-dev(a)lists.jboss.org
> >> Sent: Monday, 13 July, 2015 3:17:16 PM
> >> Subject: [keycloak-dev] Operational monitoring of Keycloak server
> >>
> >> Hi,
> >>
> >> as we deployed KC to production mode for
https://developers.redhat.com
> >> we started to think about operational monitoring, for example from
> >> Nagios or other systems of this type.
> >>
> >> KC user guide doesn't contain any chapter covering this topic, also no
> >> any success over google search, so looks like KC doesn't have any
> >> solution for this yet.
> >> But I believe this is an important area which must be solved when KC
is
> >> used for production.
> >>
> >> I can imagine monitoring of JDBC connection if JPA is used, monitoring
> >> of Mongo connection if used as store, monitoring of LDAP connection if
> >> LDAP federation is used etc.
> >> Also some statistics like numbers of active sso session, number of
> >> logins per minute etc should be provided there.
> >>
> >> Monitoring is not about Keycloak core itself, it should be available
for
> >> extension developers also. For example we implemented own
> >> UserFederationProvider which calls backend REST services.
> >> We should be able to add info about this integration into monitoring
> >> endpoint to be able to catch problems with this REST API.
> >>
> >> It should be probably implemented same way as used by underlying
> >> WildFly/EAP (JPA/JDBC is probably available for monitoring there). I'm
> >> not sure if JMX is used there still or if some new framework is
> >> available for it.
> >> Or KC should use some form of KC REST API for this, which should be
> >> extended by additional info from KC extensions?
> >>
> >> What do you think?
> >>
> >> Vlastimil
> >>
> >> P.S we have https://issues.jboss.org/browse/RHD-552 for Red Hat
> >> Developer instance of KC
> >>
> >> --
> >> Vlastimil Elias
> >> Principal Software Engineer
> >> jboss.org Development Team
> >>
> >> _______________________________________________
> >> keycloak-dev mailing list
> >> keycloak-dev(a)lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>
>
> --
> Vlastimil Elias
> Principal Software Engineer
> jboss.org Development Team
>
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
1:34 a.m.
Yap, beside server-stat endpoint Stian talked about we should add some
server-health endpoint also to be used for remote health monitoring by
load balancers or systems like Nagios.
Simple http endpoint without authentication (as it doesn't leak any
sensitive information) is typically simplest way how to do this monitoring.
Vl.
On 14.7.2015 05:13, Scott Rossillo wrote:
Some type of health page would be great too for load balancers to
monitor. Something that doesn't leak internal information but checks
behind the scenes that:
1. Server can reach its databas(es)
2. Server cluster sync is working
3. Server can reach federation providers, etc.
Endpoint should respond to get requests and return an http status
reflective of server state.
On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen <stian(a)redhat.com
<mailto:stian@redhat.com>> wrote:
So looks like we're at agreement to add the additional info you
wanted to server info page.
How about we add an additional endpoint server-stat that can
collect some stats about the server?
----- Original Message -----
> From: "Vlastimil Elias" <velias(a)redhat.com
<mailto:velias@redhat.com>>
> To: "Stian Thorgersen" <stian(a)redhat.com
<mailto:stian@redhat.com>>
> Cc: keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
> Sent: Monday, 13 July, 2015 5:06:34 PM
> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak
server
>
> Looks like I have to look at WildFly/EAP DMR to see what is
possible to
> do with it, as I'm not sure if it is about remote monitoring
also and
> if/how it can be use from monitoring systems like Splunk.
>
> Vl.
>
> On 13.7.2015 15:26, Stian Thorgersen wrote:
> > In WildFly/EAP that's DMR right? We're planning to make
Keycloak managable
> > through that as well. For example everything that goes into
> > keycloak-server.json will eventually be moved to
standalone.xml. Same with
> > admin endpoints, everything you can do there you'll eventually
be able to
> > do through DMR and jboss-cli as well.
> >
> > However, IMO it would make sense to at least expose Keycloak
specific
> > information through the admin endpoints and console as well.
Such number
> > of sessions, etc..
> >
> > ----- Original Message -----
> >> From: "Vlastimil Elias" <velias(a)redhat.com
<mailto:velias@redhat.com>>
> >> To: keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
> >> Sent: Monday, 13 July, 2015 3:17:16 PM
> >> Subject: [keycloak-dev] Operational monitoring of Keycloak server
> >>
> >> Hi,
> >>
> >> as we deployed KC to production mode for
https://developers.redhat.com
> >> we started to think about operational monitoring, for example
from
> >> Nagios or other systems of this type.
> >>
> >> KC user guide doesn't contain any chapter covering this
topic, also no
> >> any success over google search, so looks like KC doesn't have any
> >> solution for this yet.
> >> But I believe this is an important area which must be solved
when KC is
> >> used for production.
> >>
> >> I can imagine monitoring of JDBC connection if JPA is used,
monitoring
> >> of Mongo connection if used as store, monitoring of LDAP
connection if
> >> LDAP federation is used etc.
> >> Also some statistics like numbers of active sso session,
number of
> >> logins per minute etc should be provided there.
> >>
> >> Monitoring is not about Keycloak core itself, it should be
available for
> >> extension developers also. For example we implemented own
> >> UserFederationProvider which calls backend REST services.
> >> We should be able to add info about this integration into
monitoring
> >> endpoint to be able to catch problems with this REST API.
> >>
> >> It should be probably implemented same way as used by underlying
> >> WildFly/EAP (JPA/JDBC is probably available for monitoring
there). I'm
> >> not sure if JMX is used there still or if some new framework is
> >> available for it.
> >> Or KC should use some form of KC REST API for this, which
should be
> >> extended by additional info from KC extensions?
> >>
> >> What do you think?
> >>
> >> Vlastimil
> >>
> >> P.S we have https://issues.jboss.org/browse/RHD-552 for Red Hat
> >> Developer instance of KC
> >>
> >> --
> >> Vlastimil Elias
> >> Principal Software Engineer
> >> jboss.org <http://jboss.org> Development Team
> >>
> >> _______________________________________________
> >> keycloak-dev mailing list
> >> keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
> >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>
>
> --
> Vlastimil Elias
> Principal Software Engineer
> jboss.org <http://jboss.org> Development Team
>
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
1:42 a.m.
Ok so we add one that just returns OK or ERROR which doesn't require authentication,
but then we add another that can return more details, including stats and what the error
is.
For full health we should check all external services that are used (db, identity
providers and user federation providers). Cluster sync will depend on what Infinispan
exposes, but I imagine they'll have the required details avail.
----- Original Message -----
From: "Vlastimil Elias" <velias(a)redhat.com>
To: "Scott Rossillo" <srossillo(a)smartling.com>, "Stian
Thorgersen" <stian(a)redhat.com>
Cc: keycloak-dev(a)lists.jboss.org
Sent: Tuesday, 14 July, 2015 8:34:29 AM
Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
Yap, beside server-stat endpoint Stian talked about we should add some
server-health endpoint also to be used for remote health monitoring by
load balancers or systems like Nagios.
Simple http endpoint without authentication (as it doesn't leak any
sensitive information) is typically simplest way how to do this monitoring.
Vl.
On 14.7.2015 05:13, Scott Rossillo wrote:
> Some type of health page would be great too for load balancers to
> monitor. Something that doesn't leak internal information but checks
> behind the scenes that:
> 1. Server can reach its databas(es)
> 2. Server cluster sync is working
> 3. Server can reach federation providers, etc.
> Endpoint should respond to get requests and return an http status
> reflective of server state.
>
> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen <stian(a)redhat.com
> <mailto:stian@redhat.com>> wrote:
>
> So looks like we're at agreement to add the additional info you
> wanted to server info page.
>
> How about we add an additional endpoint server-stat that can
> collect some stats about the server?
>
> ----- Original Message -----
> > From: "Vlastimil Elias" <velias(a)redhat.com
> <mailto:velias@redhat.com>>
> > To: "Stian Thorgersen" <stian(a)redhat.com
<mailto:stian@redhat.com>>
> > Cc: keycloak-dev(a)lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>
> > Sent: Monday, 13 July, 2015 5:06:34 PM
> > Subject: Re: [keycloak-dev] Operational monitoring of Keycloak
> server
> >
> > Looks like I have to look at WildFly/EAP DMR to see what is
> possible to
> > do with it, as I'm not sure if it is about remote monitoring
> also and
> > if/how it can be use from monitoring systems like Splunk.
> >
> > Vl.
> >
> > On 13.7.2015 15:26, Stian Thorgersen wrote:
> > > In WildFly/EAP that's DMR right? We're planning to make
> Keycloak managable
> > > through that as well. For example everything that goes into
> > > keycloak-server.json will eventually be moved to
> standalone.xml. Same with
> > > admin endpoints, everything you can do there you'll eventually
> be able to
> > > do through DMR and jboss-cli as well.
> > >
> > > However, IMO it would make sense to at least expose Keycloak
> specific
> > > information through the admin endpoints and console as well.
> Such number
> > > of sessions, etc..
> > >
> > > ----- Original Message -----
> > >> From: "Vlastimil Elias" <velias(a)redhat.com
> <mailto:velias@redhat.com>>
> > >> To: keycloak-dev(a)lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>
> > >> Sent: Monday, 13 July, 2015 3:17:16 PM
> > >> Subject: [keycloak-dev] Operational monitoring of Keycloak server
> > >>
> > >> Hi,
> > >>
> > >> as we deployed KC to production mode for
> https://developers.redhat.com
> > >> we started to think about operational monitoring, for example
> from
> > >> Nagios or other systems of this type.
> > >>
> > >> KC user guide doesn't contain any chapter covering this
> topic, also no
> > >> any success over google search, so looks like KC doesn't have
any
> > >> solution for this yet.
> > >> But I believe this is an important area which must be solved
> when KC is
> > >> used for production.
> > >>
> > >> I can imagine monitoring of JDBC connection if JPA is used,
> monitoring
> > >> of Mongo connection if used as store, monitoring of LDAP
> connection if
> > >> LDAP federation is used etc.
> > >> Also some statistics like numbers of active sso session,
> number of
> > >> logins per minute etc should be provided there.
> > >>
> > >> Monitoring is not about Keycloak core itself, it should be
> available for
> > >> extension developers also. For example we implemented own
> > >> UserFederationProvider which calls backend REST services.
> > >> We should be able to add info about this integration into
> monitoring
> > >> endpoint to be able to catch problems with this REST API.
> > >>
> > >> It should be probably implemented same way as used by underlying
> > >> WildFly/EAP (JPA/JDBC is probably available for monitoring
> there). I'm
> > >> not sure if JMX is used there still or if some new framework is
> > >> available for it.
> > >> Or KC should use some form of KC REST API for this, which
> should be
> > >> extended by additional info from KC extensions?
> > >>
> > >> What do you think?
> > >>
> > >> Vlastimil
> > >>
> > >> P.S we have https://issues.jboss.org/browse/RHD-552 for Red Hat
> > >> Developer instance of KC
> > >>
> > >> --
> > >> Vlastimil Elias
> > >> Principal Software Engineer
> > >> jboss.org <http://jboss.org> Development Team
> > >>
> > >> _______________________________________________
> > >> keycloak-dev mailing list
> > >> keycloak-dev(a)lists.jboss.org
> <mailto:keycloak-dev@lists.jboss.org>
> > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > >>
> >
> > --
> > Vlastimil Elias
> > Principal Software Engineer
> > jboss.org <http://jboss.org> Development Team
> >
> >
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
1:56 a.m.
On 14.7.2015 08:42, Stian Thorgersen wrote:
Ok so we add one that just returns OK or ERROR which doesn't
require authentication, but then we add another that can return more details, including
stats and what the error is.
Yep. Simple http endpoinds for now, later all this info
may be expose
over DMR or some other mechanism if necessary.
For full health we should check all external services that are used (db, identity
providers and user federation providers). Cluster sync will depend on what Infinispan
exposes, but I imagine they'll have the required details avail.
Beside endpoint for full health check (which should summarize all
services into one flag if I understand correctly, which is good for
loadbalancer) we should expose separate endpoint for each service
health, as it is better for operational monitoring systems like Nagios.
Admins are better informed what is going wrong and they can resolve
situation more quickly without necessity to search cause of general
failure flag somewhere else.
Vl.
----- Original Message -----
> From: "Vlastimil Elias" <velias(a)redhat.com>
> To: "Scott Rossillo" <srossillo(a)smartling.com>, "Stian
Thorgersen" <stian(a)redhat.com>
> Cc: keycloak-dev(a)lists.jboss.org
> Sent: Tuesday, 14 July, 2015 8:34:29 AM
> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>
> Yap, beside server-stat endpoint Stian talked about we should add some
> server-health endpoint also to be used for remote health monitoring by
> load balancers or systems like Nagios.
> Simple http endpoint without authentication (as it doesn't leak any
> sensitive information) is typically simplest way how to do this monitoring.
>
> Vl.
>
> On 14.7.2015 05:13, Scott Rossillo wrote:
>> Some type of health page would be great too for load balancers to
>> monitor. Something that doesn't leak internal information but checks
>> behind the scenes that:
>> 1. Server can reach its databas(es)
>> 2. Server cluster sync is working
>> 3. Server can reach federation providers, etc.
>> Endpoint should respond to get requests and return an http status
>> reflective of server state.
>>
>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen <stian(a)redhat.com
>> <mailto:stian@redhat.com>> wrote:
>>
>> So looks like we're at agreement to add the additional info you
>> wanted to server info page.
>>
>> How about we add an additional endpoint server-stat that can
>> collect some stats about the server?
>>
>> ----- Original Message -----
>> > From: "Vlastimil Elias" <velias(a)redhat.com
>> <mailto:velias@redhat.com>>
>> > To: "Stian Thorgersen" <stian(a)redhat.com
<mailto:stian@redhat.com>>
>> > Cc: keycloak-dev(a)lists.jboss.org
>> <mailto:keycloak-dev@lists.jboss.org>
>> > Sent: Monday, 13 July, 2015 5:06:34 PM
>> > Subject: Re: [keycloak-dev] Operational monitoring of Keycloak
>> server
>> >
>> > Looks like I have to look at WildFly/EAP DMR to see what is
>> possible to
>> > do with it, as I'm not sure if it is about remote monitoring
>> also and
>> > if/how it can be use from monitoring systems like Splunk.
>> >
>> > Vl.
>> >
>> > On 13.7.2015 15:26, Stian Thorgersen wrote:
>> > > In WildFly/EAP that's DMR right? We're planning to make
>> Keycloak managable
>> > > through that as well. For example everything that goes into
>> > > keycloak-server.json will eventually be moved to
>> standalone.xml. Same with
>> > > admin endpoints, everything you can do there you'll
eventually
>> be able to
>> > > do through DMR and jboss-cli as well.
>> > >
>> > > However, IMO it would make sense to at least expose Keycloak
>> specific
>> > > information through the admin endpoints and console as well.
>> Such number
>> > > of sessions, etc..
>> > >
>> > > ----- Original Message -----
>> > >> From: "Vlastimil Elias" <velias(a)redhat.com
>> <mailto:velias@redhat.com>>
>> > >> To: keycloak-dev(a)lists.jboss.org
>> <mailto:keycloak-dev@lists.jboss.org>
>> > >> Sent: Monday, 13 July, 2015 3:17:16 PM
>> > >> Subject: [keycloak-dev] Operational monitoring of Keycloak
server
>> > >>
>> > >> Hi,
>> > >>
>> > >> as we deployed KC to production mode for
>> https://developers.redhat.com
>> > >> we started to think about operational monitoring, for example
>> from
>> > >> Nagios or other systems of this type.
>> > >>
>> > >> KC user guide doesn't contain any chapter covering this
>> topic, also no
>> > >> any success over google search, so looks like KC doesn't
have any
>> > >> solution for this yet.
>> > >> But I believe this is an important area which must be solved
>> when KC is
>> > >> used for production.
>> > >>
>> > >> I can imagine monitoring of JDBC connection if JPA is used,
>> monitoring
>> > >> of Mongo connection if used as store, monitoring of LDAP
>> connection if
>> > >> LDAP federation is used etc.
>> > >> Also some statistics like numbers of active sso session,
>> number of
>> > >> logins per minute etc should be provided there.
>> > >>
>> > >> Monitoring is not about Keycloak core itself, it should be
>> available for
>> > >> extension developers also. For example we implemented own
>> > >> UserFederationProvider which calls backend REST services.
>> > >> We should be able to add info about this integration into
>> monitoring
>> > >> endpoint to be able to catch problems with this REST API.
>> > >>
>> > >> It should be probably implemented same way as used by
underlying
>> > >> WildFly/EAP (JPA/JDBC is probably available for monitoring
>> there). I'm
>> > >> not sure if JMX is used there still or if some new framework
is
>> > >> available for it.
>> > >> Or KC should use some form of KC REST API for this, which
>> should be
>> > >> extended by additional info from KC extensions?
>> > >>
>> > >> What do you think?
>> > >>
>> > >> Vlastimil
>> > >>
>> > >> P.S we have https://issues.jboss.org/browse/RHD-552 for Red
Hat
>> > >> Developer instance of KC
>> > >>
>> > >> --
>> > >> Vlastimil Elias
>> > >> Principal Software Engineer
>> > >> jboss.org <http://jboss.org> Development Team
>> > >>
>> > >> _______________________________________________
>> > >> keycloak-dev mailing list
>> > >> keycloak-dev(a)lists.jboss.org
>> <mailto:keycloak-dev@lists.jboss.org>
>> > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>> > >>
>> >
>> > --
>> > Vlastimil Elias
>> > Principal Software Engineer
>> > jboss.org <http://jboss.org> Development Team
>> >
>> >
>> _______________________________________________
>> keycloak-dev mailing list
>> keycloak-dev(a)lists.jboss.org <mailto:keycloak-dev@lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>
> --
> Vlastimil Elias
> Principal Software Engineer
> jboss.org Development Team
>
>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
2:14 a.m.
----- Original Message -----
From: "Vlastimil Elias" <velias(a)redhat.com>
To: "Stian Thorgersen" <stian(a)redhat.com>
Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
keycloak-dev(a)lists.jboss.org
Sent: Tuesday, 14 July, 2015 8:56:43 AM
Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
On 14.7.2015 08:42, Stian Thorgersen wrote:
> Ok so we add one that just returns OK or ERROR which doesn't require
> authentication, but then we add another that can return more details,
> including stats and what the error is.
Yep. Simple http endpoinds for now, later all this info may be expose
over DMR or some other mechanism if necessary.
>
> For full health we should check all external services that are used (db,
> identity providers and user federation providers). Cluster sync will
> depend on what Infinispan exposes, but I imagine they'll have the required
> details avail.
Beside endpoint for full health check (which should summarize all
services into one flag if I understand correctly, which is good for
loadbalancer) we should expose separate endpoint for each service
health, as it is better for operational monitoring systems like Nagios.
Admins are better informed what is going wrong and they can resolve
situation more quickly without necessity to search cause of general
failure flag somewhere else.
We could have a single health endpoint that returns a 200 or 500 and also returns a json
doc with details. Something like:
{
realm: ok,
users: ok,
realmCache: error,
userCache: error,
identity-providers: {
google: ok,
facebook: error
},
user-federation: {
ldap: ok
}
}
I wonder if that's to much details to expose without authentication.
Vl.
>
> ----- Original Message -----
>> From: "Vlastimil Elias" <velias(a)redhat.com>
>> To: "Scott Rossillo" <srossillo(a)smartling.com>, "Stian
Thorgersen"
>> <stian(a)redhat.com>
>> Cc: keycloak-dev(a)lists.jboss.org
>> Sent: Tuesday, 14 July, 2015 8:34:29 AM
>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>
>> Yap, beside server-stat endpoint Stian talked about we should add some
>> server-health endpoint also to be used for remote health monitoring by
>> load balancers or systems like Nagios.
>> Simple http endpoint without authentication (as it doesn't leak any
>> sensitive information) is typically simplest way how to do this
>> monitoring.
>>
>> Vl.
>>
>> On 14.7.2015 05:13, Scott Rossillo wrote:
>>> Some type of health page would be great too for load balancers to
>>> monitor. Something that doesn't leak internal information but checks
>>> behind the scenes that:
>>> 1. Server can reach its databas(es)
>>> 2. Server cluster sync is working
>>> 3. Server can reach federation providers, etc.
>>> Endpoint should respond to get requests and return an http status
>>> reflective of server state.
>>>
>>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen <stian(a)redhat.com
>>> <mailto:stian@redhat.com>> wrote:
>>>
>>> So looks like we're at agreement to add the additional info you
>>> wanted to server info page.
>>>
>>> How about we add an additional endpoint server-stat that can
>>> collect some stats about the server?
>>>
>>> ----- Original Message -----
>>> > From: "Vlastimil Elias" <velias(a)redhat.com
>>> <mailto:velias@redhat.com>>
>>> > To: "Stian Thorgersen" <stian(a)redhat.com
>>> > <mailto:stian@redhat.com>>
>>> > Cc: keycloak-dev(a)lists.jboss.org
>>> <mailto:keycloak-dev@lists.jboss.org>
>>> > Sent: Monday, 13 July, 2015 5:06:34 PM
>>> > Subject: Re: [keycloak-dev] Operational monitoring of Keycloak
>>> server
>>> >
>>> > Looks like I have to look at WildFly/EAP DMR to see what is
>>> possible to
>>> > do with it, as I'm not sure if it is about remote monitoring
>>> also and
>>> > if/how it can be use from monitoring systems like Splunk.
>>> >
>>> > Vl.
>>> >
>>> > On 13.7.2015 15:26, Stian Thorgersen wrote:
>>> > > In WildFly/EAP that's DMR right? We're planning to
make
>>> Keycloak managable
>>> > > through that as well. For example everything that goes into
>>> > > keycloak-server.json will eventually be moved to
>>> standalone.xml. Same with
>>> > > admin endpoints, everything you can do there you'll
eventually
>>> be able to
>>> > > do through DMR and jboss-cli as well.
>>> > >
>>> > > However, IMO it would make sense to at least expose Keycloak
>>> specific
>>> > > information through the admin endpoints and console as well.
>>> Such number
>>> > > of sessions, etc..
>>> > >
>>> > > ----- Original Message -----
>>> > >> From: "Vlastimil Elias" <velias(a)redhat.com
>>> <mailto:velias@redhat.com>>
>>> > >> To: keycloak-dev(a)lists.jboss.org
>>> <mailto:keycloak-dev@lists.jboss.org>
>>> > >> Sent: Monday, 13 July, 2015 3:17:16 PM
>>> > >> Subject: [keycloak-dev] Operational monitoring of
Keycloak
>>> > >> server
>>> > >>
>>> > >> Hi,
>>> > >>
>>> > >> as we deployed KC to production mode for
>>> https://developers.redhat.com
>>> > >> we started to think about operational monitoring, for
example
>>> from
>>> > >> Nagios or other systems of this type.
>>> > >>
>>> > >> KC user guide doesn't contain any chapter covering
this
>>> topic, also no
>>> > >> any success over google search, so looks like KC
doesn't have
>>> > >> any
>>> > >> solution for this yet.
>>> > >> But I believe this is an important area which must be
solved
>>> when KC is
>>> > >> used for production.
>>> > >>
>>> > >> I can imagine monitoring of JDBC connection if JPA is
used,
>>> monitoring
>>> > >> of Mongo connection if used as store, monitoring of LDAP
>>> connection if
>>> > >> LDAP federation is used etc.
>>> > >> Also some statistics like numbers of active sso session,
>>> number of
>>> > >> logins per minute etc should be provided there.
>>> > >>
>>> > >> Monitoring is not about Keycloak core itself, it should
be
>>> available for
>>> > >> extension developers also. For example we implemented
own
>>> > >> UserFederationProvider which calls backend REST
services.
>>> > >> We should be able to add info about this integration
into
>>> monitoring
>>> > >> endpoint to be able to catch problems with this REST
API.
>>> > >>
>>> > >> It should be probably implemented same way as used by
>>> > >> underlying
>>> > >> WildFly/EAP (JPA/JDBC is probably available for
monitoring
>>> there). I'm
>>> > >> not sure if JMX is used there still or if some new
framework is
>>> > >> available for it.
>>> > >> Or KC should use some form of KC REST API for this,
which
>>> should be
>>> > >> extended by additional info from KC extensions?
>>> > >>
>>> > >> What do you think?
>>> > >>
>>> > >> Vlastimil
>>> > >>
>>> > >> P.S we have https://issues.jboss.org/browse/RHD-552 for
Red Hat
>>> > >> Developer instance of KC
>>> > >>
>>> > >> --
>>> > >> Vlastimil Elias
>>> > >> Principal Software Engineer
>>> > >> jboss.org <http://jboss.org> Development Team
>>> > >>
>>> > >> _______________________________________________
>>> > >> keycloak-dev mailing list
>>> > >> keycloak-dev(a)lists.jboss.org
>>> <mailto:keycloak-dev@lists.jboss.org>
>>> > >> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>> > >>
>>> >
>>> > --
>>> > Vlastimil Elias
>>> > Principal Software Engineer
>>> > jboss.org <http://jboss.org> Development Team
>>> >
>>> >
>>> _______________________________________________
>>> keycloak-dev mailing list
>>> keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>
>> --
>> Vlastimil Elias
>> Principal Software Engineer
>> jboss.org Development Team
>>
>>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
2:58 a.m.
On 14.7.2015 09:14, Stian Thorgersen wrote:
----- Original Message -----
> From: "Vlastimil Elias" <velias(a)redhat.com>
> To: "Stian Thorgersen" <stian(a)redhat.com>
> Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
keycloak-dev(a)lists.jboss.org
> Sent: Tuesday, 14 July, 2015 8:56:43 AM
> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>
>
>
> On 14.7.2015 08:42, Stian Thorgersen wrote:
>> Ok so we add one that just returns OK or ERROR which doesn't require
>> authentication, but then we add another that can return more details,
>> including stats and what the error is.
> Yep. Simple http endpoinds for now, later all this info may be expose
> over DMR or some other mechanism if necessary.
>> For full health we should check all external services that are used (db,
>> identity providers and user federation providers). Cluster sync will
>> depend on what Infinispan exposes, but I imagine they'll have the required
>> details avail.
> Beside endpoint for full health check (which should summarize all
> services into one flag if I understand correctly, which is good for
> loadbalancer) we should expose separate endpoint for each service
> health, as it is better for operational monitoring systems like Nagios.
> Admins are better informed what is going wrong and they can resolve
> situation more quickly without necessity to search cause of general
> failure flag somewhere else.
We could have a single health endpoint that returns a 200 or 500 and also returns a json
doc with details. Something like:
{
realm: ok,
users: ok,
realmCache: error,
userCache: error,
identity-providers: {
google: ok,
facebook: error
},
user-federation: {
ldap: ok
}
}
I wonder if that's to much details to expose without authentication.
This should work, but I miss info about DB connection here. It is
crucial from operational point of view as this allows sysadmins quickly
detect that connection to DB is broken.
I believe this kind of info can be safely exposed without
authentication. If the endpoint is clearly documented then access can be
always restricted on some network infra (webserver/proxy) ahead of KC.
> Vl.
>
>> ----- Original Message -----
>>> From: "Vlastimil Elias" <velias(a)redhat.com>
>>> To: "Scott Rossillo" <srossillo(a)smartling.com>, "Stian
Thorgersen"
>>> <stian(a)redhat.com>
>>> Cc: keycloak-dev(a)lists.jboss.org
>>> Sent: Tuesday, 14 July, 2015 8:34:29 AM
>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>>
>>> Yap, beside server-stat endpoint Stian talked about we should add some
>>> server-health endpoint also to be used for remote health monitoring by
>>> load balancers or systems like Nagios.
>>> Simple http endpoint without authentication (as it doesn't leak any
>>> sensitive information) is typically simplest way how to do this
>>> monitoring.
>>>
>>> Vl.
>>>
>>> On 14.7.2015 05:13, Scott Rossillo wrote:
>>>> Some type of health page would be great too for load balancers to
>>>> monitor. Something that doesn't leak internal information but checks
>>>> behind the scenes that:
>>>> 1. Server can reach its databas(es)
>>>> 2. Server cluster sync is working
>>>> 3. Server can reach federation providers, etc.
>>>> Endpoint should respond to get requests and return an http status
>>>> reflective of server state.
>>>>
>>>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen <stian(a)redhat.com
>>>> <mailto:stian@redhat.com>> wrote:
>>>>
>>>> So looks like we're at agreement to add the additional info
you
>>>> wanted to server info page.
>>>>
>>>> How about we add an additional endpoint server-stat that can
>>>> collect some stats about the server?
>>>>
>>>> ----- Original Message -----
>>>> > From: "Vlastimil Elias" <velias(a)redhat.com
>>>> <mailto:velias@redhat.com>>
>>>> > To: "Stian Thorgersen" <stian(a)redhat.com
>>>> > <mailto:stian@redhat.com>>
>>>> > Cc: keycloak-dev(a)lists.jboss.org
>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>> > Sent: Monday, 13 July, 2015 5:06:34 PM
>>>> > Subject: Re: [keycloak-dev] Operational monitoring of
Keycloak
>>>> server
>>>> >
>>>> > Looks like I have to look at WildFly/EAP DMR to see what is
>>>> possible to
>>>> > do with it, as I'm not sure if it is about remote
monitoring
>>>> also and
>>>> > if/how it can be use from monitoring systems like Splunk.
>>>> >
>>>> > Vl.
>>>> >
>>>> > On 13.7.2015 15:26, Stian Thorgersen wrote:
>>>> > > In WildFly/EAP that's DMR right? We're planning
to make
>>>> Keycloak managable
>>>> > > through that as well. For example everything that goes
into
>>>> > > keycloak-server.json will eventually be moved to
>>>> standalone.xml. Same with
>>>> > > admin endpoints, everything you can do there you'll
eventually
>>>> be able to
>>>> > > do through DMR and jboss-cli as well.
>>>> > >
>>>> > > However, IMO it would make sense to at least expose
Keycloak
>>>> specific
>>>> > > information through the admin endpoints and console as
well.
>>>> Such number
>>>> > > of sessions, etc..
>>>> > >
>>>> > > ----- Original Message -----
>>>> > >> From: "Vlastimil Elias"
<velias(a)redhat.com
>>>> <mailto:velias@redhat.com>>
>>>> > >> To: keycloak-dev(a)lists.jboss.org
>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>> > >> Sent: Monday, 13 July, 2015 3:17:16 PM
>>>> > >> Subject: [keycloak-dev] Operational monitoring of
Keycloak
>>>> > >> server
>>>> > >>
>>>> > >> Hi,
>>>> > >>
>>>> > >> as we deployed KC to production mode for
>>>> https://developers.redhat.com
>>>> > >> we started to think about operational monitoring, for
example
>>>> from
>>>> > >> Nagios or other systems of this type.
>>>> > >>
>>>> > >> KC user guide doesn't contain any chapter
covering this
>>>> topic, also no
>>>> > >> any success over google search, so looks like KC
doesn't have
>>>> > >> any
>>>> > >> solution for this yet.
>>>> > >> But I believe this is an important area which must be
solved
>>>> when KC is
>>>> > >> used for production.
>>>> > >>
>>>> > >> I can imagine monitoring of JDBC connection if JPA is
used,
>>>> monitoring
>>>> > >> of Mongo connection if used as store, monitoring of
LDAP
>>>> connection if
>>>> > >> LDAP federation is used etc.
>>>> > >> Also some statistics like numbers of active sso
session,
>>>> number of
>>>> > >> logins per minute etc should be provided there.
>>>> > >>
>>>> > >> Monitoring is not about Keycloak core itself, it
should be
>>>> available for
>>>> > >> extension developers also. For example we implemented
own
>>>> > >> UserFederationProvider which calls backend REST
services.
>>>> > >> We should be able to add info about this integration
into
>>>> monitoring
>>>> > >> endpoint to be able to catch problems with this REST
API.
>>>> > >>
>>>> > >> It should be probably implemented same way as used
by
>>>> > >> underlying
>>>> > >> WildFly/EAP (JPA/JDBC is probably available for
monitoring
>>>> there). I'm
>>>> > >> not sure if JMX is used there still or if some new
framework is
>>>> > >> available for it.
>>>> > >> Or KC should use some form of KC REST API for this,
which
>>>> should be
>>>> > >> extended by additional info from KC extensions?
>>>> > >>
>>>> > >> What do you think?
>>>> > >>
>>>> > >> Vlastimil
>>>> > >>
>>>> > >> P.S we have https://issues.jboss.org/browse/RHD-552
for Red Hat
>>>> > >> Developer instance of KC
>>>> > >>
>>>> > >> --
>>>> > >> Vlastimil Elias
>>>> > >> Principal Software Engineer
>>>> > >> jboss.org <http://jboss.org> Development Team
>>>> > >>
>>>> > >> _______________________________________________
>>>> > >> keycloak-dev mailing list
>>>> > >> keycloak-dev(a)lists.jboss.org
>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>> > >>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>> > >>
>>>> >
>>>> > --
>>>> > Vlastimil Elias
>>>> > Principal Software Engineer
>>>> > jboss.org <http://jboss.org> Development Team
>>>> >
>>>> >
>>>> _______________________________________________
>>>> keycloak-dev mailing list
>>>> keycloak-dev(a)lists.jboss.org
<mailto:keycloak-dev@lists.jboss.org>
>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>
>>> --
>>> Vlastimil Elias
>>> Principal Software Engineer
>>> jboss.org Development Team
>>>
>>>
> --
> Vlastimil Elias
> Principal Software Engineer
> jboss.org Development Team
>
>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
3:06 a.m.
----- Original Message -----
From: "Vlastimil Elias" <velias(a)redhat.com>
To: "Stian Thorgersen" <stian(a)redhat.com>
Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
keycloak-dev(a)lists.jboss.org
Sent: Tuesday, 14 July, 2015 9:58:36 AM
Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
On 14.7.2015 09:14, Stian Thorgersen wrote:
>
> ----- Original Message -----
>> From: "Vlastimil Elias" <velias(a)redhat.com>
>> To: "Stian Thorgersen" <stian(a)redhat.com>
>> Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
>> keycloak-dev(a)lists.jboss.org
>> Sent: Tuesday, 14 July, 2015 8:56:43 AM
>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>
>>
>>
>> On 14.7.2015 08:42, Stian Thorgersen wrote:
>>> Ok so we add one that just returns OK or ERROR which doesn't require
>>> authentication, but then we add another that can return more details,
>>> including stats and what the error is.
>> Yep. Simple http endpoinds for now, later all this info may be expose
>> over DMR or some other mechanism if necessary.
>>> For full health we should check all external services that are used (db,
>>> identity providers and user federation providers). Cluster sync will
>>> depend on what Infinispan exposes, but I imagine they'll have the
>>> required
>>> details avail.
>> Beside endpoint for full health check (which should summarize all
>> services into one flag if I understand correctly, which is good for
>> loadbalancer) we should expose separate endpoint for each service
>> health, as it is better for operational monitoring systems like Nagios.
>> Admins are better informed what is going wrong and they can resolve
>> situation more quickly without necessity to search cause of general
>> failure flag somewhere else.
> We could have a single health endpoint that returns a 200 or 500 and also
> returns a json doc with details. Something like:
>
> {
> realm: ok,
> users: ok,
> realmCache: error,
> userCache: error,
> identity-providers: {
> google: ok,
> facebook: error
> },
> user-federation: {
> ldap: ok
> }
> }
>
> I wonder if that's to much details to expose without authentication.
This should work, but I miss info about DB connection here. It is
crucial from operational point of view as this allows sysadmins quickly
detect that connection to DB is broken.
I believe this kind of info can be safely exposed without
authentication. If the endpoint is clearly documented then access can be
always restricted on some network infra (webserver/proxy) ahead of KC.
DB is "realm" and "users".
>
>> Vl.
>>
>>> ----- Original Message -----
>>>> From: "Vlastimil Elias" <velias(a)redhat.com>
>>>> To: "Scott Rossillo" <srossillo(a)smartling.com>,
"Stian Thorgersen"
>>>> <stian(a)redhat.com>
>>>> Cc: keycloak-dev(a)lists.jboss.org
>>>> Sent: Tuesday, 14 July, 2015 8:34:29 AM
>>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>>>
>>>> Yap, beside server-stat endpoint Stian talked about we should add some
>>>> server-health endpoint also to be used for remote health monitoring by
>>>> load balancers or systems like Nagios.
>>>> Simple http endpoint without authentication (as it doesn't leak any
>>>> sensitive information) is typically simplest way how to do this
>>>> monitoring.
>>>>
>>>> Vl.
>>>>
>>>> On 14.7.2015 05:13, Scott Rossillo wrote:
>>>>> Some type of health page would be great too for load balancers to
>>>>> monitor. Something that doesn't leak internal information but
checks
>>>>> behind the scenes that:
>>>>> 1. Server can reach its databas(es)
>>>>> 2. Server cluster sync is working
>>>>> 3. Server can reach federation providers, etc.
>>>>> Endpoint should respond to get requests and return an http status
>>>>> reflective of server state.
>>>>>
>>>>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen
<stian(a)redhat.com
>>>>> <mailto:stian@redhat.com>> wrote:
>>>>>
>>>>> So looks like we're at agreement to add the additional
info you
>>>>> wanted to server info page.
>>>>>
>>>>> How about we add an additional endpoint server-stat that can
>>>>> collect some stats about the server?
>>>>>
>>>>> ----- Original Message -----
>>>>> > From: "Vlastimil Elias" <velias(a)redhat.com
>>>>> <mailto:velias@redhat.com>>
>>>>> > To: "Stian Thorgersen" <stian(a)redhat.com
>>>>> > <mailto:stian@redhat.com>>
>>>>> > Cc: keycloak-dev(a)lists.jboss.org
>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>> > Sent: Monday, 13 July, 2015 5:06:34 PM
>>>>> > Subject: Re: [keycloak-dev] Operational monitoring of
Keycloak
>>>>> server
>>>>> >
>>>>> > Looks like I have to look at WildFly/EAP DMR to see what
is
>>>>> possible to
>>>>> > do with it, as I'm not sure if it is about remote
monitoring
>>>>> also and
>>>>> > if/how it can be use from monitoring systems like
Splunk.
>>>>> >
>>>>> > Vl.
>>>>> >
>>>>> > On 13.7.2015 15:26, Stian Thorgersen wrote:
>>>>> > > In WildFly/EAP that's DMR right? We're
planning to make
>>>>> Keycloak managable
>>>>> > > through that as well. For example everything that
goes into
>>>>> > > keycloak-server.json will eventually be moved to
>>>>> standalone.xml. Same with
>>>>> > > admin endpoints, everything you can do there
you'll
>>>>> > > eventually
>>>>> be able to
>>>>> > > do through DMR and jboss-cli as well.
>>>>> > >
>>>>> > > However, IMO it would make sense to at least expose
Keycloak
>>>>> specific
>>>>> > > information through the admin endpoints and console
as well.
>>>>> Such number
>>>>> > > of sessions, etc..
>>>>> > >
>>>>> > > ----- Original Message -----
>>>>> > >> From: "Vlastimil Elias"
<velias(a)redhat.com
>>>>> <mailto:velias@redhat.com>>
>>>>> > >> To: keycloak-dev(a)lists.jboss.org
>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>> > >> Sent: Monday, 13 July, 2015 3:17:16 PM
>>>>> > >> Subject: [keycloak-dev] Operational monitoring
of Keycloak
>>>>> > >> server
>>>>> > >>
>>>>> > >> Hi,
>>>>> > >>
>>>>> > >> as we deployed KC to production mode for
>>>>> https://developers.redhat.com
>>>>> > >> we started to think about operational
monitoring, for
>>>>> > >> example
>>>>> from
>>>>> > >> Nagios or other systems of this type.
>>>>> > >>
>>>>> > >> KC user guide doesn't contain any chapter
covering this
>>>>> topic, also no
>>>>> > >> any success over google search, so looks like KC
doesn't
>>>>> > >> have
>>>>> > >> any
>>>>> > >> solution for this yet.
>>>>> > >> But I believe this is an important area which
must be solved
>>>>> when KC is
>>>>> > >> used for production.
>>>>> > >>
>>>>> > >> I can imagine monitoring of JDBC connection if
JPA is used,
>>>>> monitoring
>>>>> > >> of Mongo connection if used as store, monitoring
of LDAP
>>>>> connection if
>>>>> > >> LDAP federation is used etc.
>>>>> > >> Also some statistics like numbers of active sso
session,
>>>>> number of
>>>>> > >> logins per minute etc should be provided there.
>>>>> > >>
>>>>> > >> Monitoring is not about Keycloak core itself, it
should be
>>>>> available for
>>>>> > >> extension developers also. For example we
implemented own
>>>>> > >> UserFederationProvider which calls backend REST
services.
>>>>> > >> We should be able to add info about this
integration into
>>>>> monitoring
>>>>> > >> endpoint to be able to catch problems with this
REST API.
>>>>> > >>
>>>>> > >> It should be probably implemented same way as
used by
>>>>> > >> underlying
>>>>> > >> WildFly/EAP (JPA/JDBC is probably available for
monitoring
>>>>> there). I'm
>>>>> > >> not sure if JMX is used there still or if some
new framework
>>>>> > >> is
>>>>> > >> available for it.
>>>>> > >> Or KC should use some form of KC REST API for
this, which
>>>>> should be
>>>>> > >> extended by additional info from KC extensions?
>>>>> > >>
>>>>> > >> What do you think?
>>>>> > >>
>>>>> > >> Vlastimil
>>>>> > >>
>>>>> > >> P.S we have
https://issues.jboss.org/browse/RHD-552 for Red
>>>>> > >> Hat
>>>>> > >> Developer instance of KC
>>>>> > >>
>>>>> > >> --
>>>>> > >> Vlastimil Elias
>>>>> > >> Principal Software Engineer
>>>>> > >> jboss.org <http://jboss.org> Development
Team
>>>>> > >>
>>>>> > >> _______________________________________________
>>>>> > >> keycloak-dev mailing list
>>>>> > >> keycloak-dev(a)lists.jboss.org
>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>> > >>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>> > >>
>>>>> >
>>>>> > --
>>>>> > Vlastimil Elias
>>>>> > Principal Software Engineer
>>>>> > jboss.org <http://jboss.org> Development Team
>>>>> >
>>>>> >
>>>>> _______________________________________________
>>>>> keycloak-dev mailing list
>>>>> keycloak-dev(a)lists.jboss.org
>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>
>>>> --
>>>> Vlastimil Elias
>>>> Principal Software Engineer
>>>> jboss.org Development Team
>>>>
>>>>
>> --
>> Vlastimil Elias
>> Principal Software Engineer
>> jboss.org Development Team
>>
>>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
1:15 a.m.
Hi,
OK, if DB is "realm" and "users" then it have to be well documented as
it is not obvious for people without deeper knowledge of KC.
Do we have JIRA issue for this or should I create one?
Vl.
On 14.7.2015 10:06, Stian Thorgersen wrote:
----- Original Message -----
> From: "Vlastimil Elias" <velias(a)redhat.com>
> To: "Stian Thorgersen" <stian(a)redhat.com>
> Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
keycloak-dev(a)lists.jboss.org
> Sent: Tuesday, 14 July, 2015 9:58:36 AM
> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>
> On 14.7.2015 09:14, Stian Thorgersen wrote:
>> ----- Original Message -----
>>> From: "Vlastimil Elias" <velias(a)redhat.com>
>>> To: "Stian Thorgersen" <stian(a)redhat.com>
>>> Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
>>> keycloak-dev(a)lists.jboss.org
>>> Sent: Tuesday, 14 July, 2015 8:56:43 AM
>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>>
>>>
>>>
>>> On 14.7.2015 08:42, Stian Thorgersen wrote:
>>>> Ok so we add one that just returns OK or ERROR which doesn't require
>>>> authentication, but then we add another that can return more details,
>>>> including stats and what the error is.
>>> Yep. Simple http endpoinds for now, later all this info may be expose
>>> over DMR or some other mechanism if necessary.
>>>> For full health we should check all external services that are used (db,
>>>> identity providers and user federation providers). Cluster sync will
>>>> depend on what Infinispan exposes, but I imagine they'll have the
>>>> required
>>>> details avail.
>>> Beside endpoint for full health check (which should summarize all
>>> services into one flag if I understand correctly, which is good for
>>> loadbalancer) we should expose separate endpoint for each service
>>> health, as it is better for operational monitoring systems like Nagios.
>>> Admins are better informed what is going wrong and they can resolve
>>> situation more quickly without necessity to search cause of general
>>> failure flag somewhere else.
>> We could have a single health endpoint that returns a 200 or 500 and also
>> returns a json doc with details. Something like:
>>
>> {
>> realm: ok,
>> users: ok,
>> realmCache: error,
>> userCache: error,
>> identity-providers: {
>> google: ok,
>> facebook: error
>> },
>> user-federation: {
>> ldap: ok
>> }
>> }
>>
>> I wonder if that's to much details to expose without authentication.
> This should work, but I miss info about DB connection here. It is
> crucial from operational point of view as this allows sysadmins quickly
> detect that connection to DB is broken.
>
> I believe this kind of info can be safely exposed without
> authentication. If the endpoint is clearly documented then access can be
> always restricted on some network infra (webserver/proxy) ahead of KC.
DB is "realm" and "users".
>>> Vl.
>>>
>>>> ----- Original Message -----
>>>>> From: "Vlastimil Elias" <velias(a)redhat.com>
>>>>> To: "Scott Rossillo" <srossillo(a)smartling.com>,
"Stian Thorgersen"
>>>>> <stian(a)redhat.com>
>>>>> Cc: keycloak-dev(a)lists.jboss.org
>>>>> Sent: Tuesday, 14 July, 2015 8:34:29 AM
>>>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak
server
>>>>>
>>>>> Yap, beside server-stat endpoint Stian talked about we should add
some
>>>>> server-health endpoint also to be used for remote health monitoring
by
>>>>> load balancers or systems like Nagios.
>>>>> Simple http endpoint without authentication (as it doesn't leak
any
>>>>> sensitive information) is typically simplest way how to do this
>>>>> monitoring.
>>>>>
>>>>> Vl.
>>>>>
>>>>> On 14.7.2015 05:13, Scott Rossillo wrote:
>>>>>> Some type of health page would be great too for load balancers
to
>>>>>> monitor. Something that doesn't leak internal information but
checks
>>>>>> behind the scenes that:
>>>>>> 1. Server can reach its databas(es)
>>>>>> 2. Server cluster sync is working
>>>>>> 3. Server can reach federation providers, etc.
>>>>>> Endpoint should respond to get requests and return an http
status
>>>>>> reflective of server state.
>>>>>>
>>>>>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen
<stian(a)redhat.com
>>>>>> <mailto:stian@redhat.com>> wrote:
>>>>>>
>>>>>> So looks like we're at agreement to add the additional
info you
>>>>>> wanted to server info page.
>>>>>>
>>>>>> How about we add an additional endpoint server-stat that
can
>>>>>> collect some stats about the server?
>>>>>>
>>>>>> ----- Original Message -----
>>>>>> > From: "Vlastimil Elias"
<velias(a)redhat.com
>>>>>> <mailto:velias@redhat.com>>
>>>>>> > To: "Stian Thorgersen"
<stian(a)redhat.com
>>>>>> > <mailto:stian@redhat.com>>
>>>>>> > Cc: keycloak-dev(a)lists.jboss.org
>>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>>> > Sent: Monday, 13 July, 2015 5:06:34 PM
>>>>>> > Subject: Re: [keycloak-dev] Operational monitoring of
Keycloak
>>>>>> server
>>>>>> >
>>>>>> > Looks like I have to look at WildFly/EAP DMR to see
what is
>>>>>> possible to
>>>>>> > do with it, as I'm not sure if it is about remote
monitoring
>>>>>> also and
>>>>>> > if/how it can be use from monitoring systems like
Splunk.
>>>>>> >
>>>>>> > Vl.
>>>>>> >
>>>>>> > On 13.7.2015 15:26, Stian Thorgersen wrote:
>>>>>> > > In WildFly/EAP that's DMR right? We're
planning to make
>>>>>> Keycloak managable
>>>>>> > > through that as well. For example everything
that goes into
>>>>>> > > keycloak-server.json will eventually be moved
to
>>>>>> standalone.xml. Same with
>>>>>> > > admin endpoints, everything you can do there
you'll
>>>>>> > > eventually
>>>>>> be able to
>>>>>> > > do through DMR and jboss-cli as well.
>>>>>> > >
>>>>>> > > However, IMO it would make sense to at least
expose Keycloak
>>>>>> specific
>>>>>> > > information through the admin endpoints and
console as well.
>>>>>> Such number
>>>>>> > > of sessions, etc..
>>>>>> > >
>>>>>> > > ----- Original Message -----
>>>>>> > >> From: "Vlastimil Elias"
<velias(a)redhat.com
>>>>>> <mailto:velias@redhat.com>>
>>>>>> > >> To: keycloak-dev(a)lists.jboss.org
>>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>>> > >> Sent: Monday, 13 July, 2015 3:17:16 PM
>>>>>> > >> Subject: [keycloak-dev] Operational
monitoring of Keycloak
>>>>>> > >> server
>>>>>> > >>
>>>>>> > >> Hi,
>>>>>> > >>
>>>>>> > >> as we deployed KC to production mode for
>>>>>> https://developers.redhat.com
>>>>>> > >> we started to think about operational
monitoring, for
>>>>>> > >> example
>>>>>> from
>>>>>> > >> Nagios or other systems of this type.
>>>>>> > >>
>>>>>> > >> KC user guide doesn't contain any
chapter covering this
>>>>>> topic, also no
>>>>>> > >> any success over google search, so looks
like KC doesn't
>>>>>> > >> have
>>>>>> > >> any
>>>>>> > >> solution for this yet.
>>>>>> > >> But I believe this is an important area
which must be solved
>>>>>> when KC is
>>>>>> > >> used for production.
>>>>>> > >>
>>>>>> > >> I can imagine monitoring of JDBC connection
if JPA is used,
>>>>>> monitoring
>>>>>> > >> of Mongo connection if used as store,
monitoring of LDAP
>>>>>> connection if
>>>>>> > >> LDAP federation is used etc.
>>>>>> > >> Also some statistics like numbers of active
sso session,
>>>>>> number of
>>>>>> > >> logins per minute etc should be provided
there.
>>>>>> > >>
>>>>>> > >> Monitoring is not about Keycloak core
itself, it should be
>>>>>> available for
>>>>>> > >> extension developers also. For example we
implemented own
>>>>>> > >> UserFederationProvider which calls backend
REST services.
>>>>>> > >> We should be able to add info about this
integration into
>>>>>> monitoring
>>>>>> > >> endpoint to be able to catch problems with
this REST API.
>>>>>> > >>
>>>>>> > >> It should be probably implemented same way
as used by
>>>>>> > >> underlying
>>>>>> > >> WildFly/EAP (JPA/JDBC is probably available
for monitoring
>>>>>> there). I'm
>>>>>> > >> not sure if JMX is used there still or if
some new framework
>>>>>> > >> is
>>>>>> > >> available for it.
>>>>>> > >> Or KC should use some form of KC REST API
for this, which
>>>>>> should be
>>>>>> > >> extended by additional info from KC
extensions?
>>>>>> > >>
>>>>>> > >> What do you think?
>>>>>> > >>
>>>>>> > >> Vlastimil
>>>>>> > >>
>>>>>> > >> P.S we have
https://issues.jboss.org/browse/RHD-552 for Red
>>>>>> > >> Hat
>>>>>> > >> Developer instance of KC
>>>>>> > >>
>>>>>> > >> --
>>>>>> > >> Vlastimil Elias
>>>>>> > >> Principal Software Engineer
>>>>>> > >> jboss.org <http://jboss.org>
Development Team
>>>>>> > >>
>>>>>> > >>
_______________________________________________
>>>>>> > >> keycloak-dev mailing list
>>>>>> > >> keycloak-dev(a)lists.jboss.org
>>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>>> > >>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>> > >>
>>>>>> >
>>>>>> > --
>>>>>> > Vlastimil Elias
>>>>>> > Principal Software Engineer
>>>>>> > jboss.org <http://jboss.org> Development Team
>>>>>> >
>>>>>> >
>>>>>> _______________________________________________
>>>>>> keycloak-dev mailing list
>>>>>> keycloak-dev(a)lists.jboss.org
>>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>>> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>
>>>>> --
>>>>> Vlastimil Elias
>>>>> Principal Software Engineer
>>>>> jboss.org Development Team
>>>>>
>>>>>
>>> --
>>> Vlastimil Elias
>>> Principal Software Engineer
>>> jboss.org Development Team
>>>
>>>
> --
> Vlastimil Elias
> Principal Software Engineer
> jboss.org Development Team
>
>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
1:49 a.m.
----- Original Message -----
From: "Vlastimil Elias" <velias(a)redhat.com>
To: "Stian Thorgersen" <stian(a)redhat.com>
Cc: keycloak-dev(a)lists.jboss.org
Sent: Thursday, 16 July, 2015 8:15:29 AM
Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
Hi,
OK, if DB is "realm" and "users" then it have to be well documented
as
it is not obvious for people without deeper knowledge of KC.
We could always call it realm-database or something, but we have 3 different stores so
should be separate status for them
Do we have JIRA issue for this or should I create one?
Nope, so please create one
Vl.
On 14.7.2015 10:06, Stian Thorgersen wrote:
>
> ----- Original Message -----
>> From: "Vlastimil Elias" <velias(a)redhat.com>
>> To: "Stian Thorgersen" <stian(a)redhat.com>
>> Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
>> keycloak-dev(a)lists.jboss.org
>> Sent: Tuesday, 14 July, 2015 9:58:36 AM
>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>
>> On 14.7.2015 09:14, Stian Thorgersen wrote:
>>> ----- Original Message -----
>>>> From: "Vlastimil Elias" <velias(a)redhat.com>
>>>> To: "Stian Thorgersen" <stian(a)redhat.com>
>>>> Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
>>>> keycloak-dev(a)lists.jboss.org
>>>> Sent: Tuesday, 14 July, 2015 8:56:43 AM
>>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>>>>
>>>>
>>>>
>>>> On 14.7.2015 08:42, Stian Thorgersen wrote:
>>>>> Ok so we add one that just returns OK or ERROR which doesn't
require
>>>>> authentication, but then we add another that can return more
details,
>>>>> including stats and what the error is.
>>>> Yep. Simple http endpoinds for now, later all this info may be expose
>>>> over DMR or some other mechanism if necessary.
>>>>> For full health we should check all external services that are used
>>>>> (db,
>>>>> identity providers and user federation providers). Cluster sync
will
>>>>> depend on what Infinispan exposes, but I imagine they'll have
the
>>>>> required
>>>>> details avail.
>>>> Beside endpoint for full health check (which should summarize all
>>>> services into one flag if I understand correctly, which is good for
>>>> loadbalancer) we should expose separate endpoint for each service
>>>> health, as it is better for operational monitoring systems like Nagios.
>>>> Admins are better informed what is going wrong and they can resolve
>>>> situation more quickly without necessity to search cause of general
>>>> failure flag somewhere else.
>>> We could have a single health endpoint that returns a 200 or 500 and also
>>> returns a json doc with details. Something like:
>>>
>>> {
>>> realm: ok,
>>> users: ok,
>>> realmCache: error,
>>> userCache: error,
>>> identity-providers: {
>>> google: ok,
>>> facebook: error
>>> },
>>> user-federation: {
>>> ldap: ok
>>> }
>>> }
>>>
>>> I wonder if that's to much details to expose without authentication.
>> This should work, but I miss info about DB connection here. It is
>> crucial from operational point of view as this allows sysadmins quickly
>> detect that connection to DB is broken.
>>
>> I believe this kind of info can be safely exposed without
>> authentication. If the endpoint is clearly documented then access can be
>> always restricted on some network infra (webserver/proxy) ahead of KC.
> DB is "realm" and "users".
>
>>>> Vl.
>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Vlastimil Elias" <velias(a)redhat.com>
>>>>>> To: "Scott Rossillo" <srossillo(a)smartling.com>,
"Stian Thorgersen"
>>>>>> <stian(a)redhat.com>
>>>>>> Cc: keycloak-dev(a)lists.jboss.org
>>>>>> Sent: Tuesday, 14 July, 2015 8:34:29 AM
>>>>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak
server
>>>>>>
>>>>>> Yap, beside server-stat endpoint Stian talked about we should
add some
>>>>>> server-health endpoint also to be used for remote health
monitoring by
>>>>>> load balancers or systems like Nagios.
>>>>>> Simple http endpoint without authentication (as it doesn't
leak any
>>>>>> sensitive information) is typically simplest way how to do this
>>>>>> monitoring.
>>>>>>
>>>>>> Vl.
>>>>>>
>>>>>> On 14.7.2015 05:13, Scott Rossillo wrote:
>>>>>>> Some type of health page would be great too for load
balancers to
>>>>>>> monitor. Something that doesn't leak internal
information but checks
>>>>>>> behind the scenes that:
>>>>>>> 1. Server can reach its databas(es)
>>>>>>> 2. Server cluster sync is working
>>>>>>> 3. Server can reach federation providers, etc.
>>>>>>> Endpoint should respond to get requests and return an http
status
>>>>>>> reflective of server state.
>>>>>>>
>>>>>>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen
<stian(a)redhat.com
>>>>>>> <mailto:stian@redhat.com>> wrote:
>>>>>>>
>>>>>>> So looks like we're at agreement to add the
additional info
>>>>>>> you
>>>>>>> wanted to server info page.
>>>>>>>
>>>>>>> How about we add an additional endpoint server-stat
that can
>>>>>>> collect some stats about the server?
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>> > From: "Vlastimil Elias"
<velias(a)redhat.com
>>>>>>> <mailto:velias@redhat.com>>
>>>>>>> > To: "Stian Thorgersen"
<stian(a)redhat.com
>>>>>>> > <mailto:stian@redhat.com>>
>>>>>>> > Cc: keycloak-dev(a)lists.jboss.org
>>>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>>>> > Sent: Monday, 13 July, 2015 5:06:34 PM
>>>>>>> > Subject: Re: [keycloak-dev] Operational
monitoring of
>>>>>>> > Keycloak
>>>>>>> server
>>>>>>> >
>>>>>>> > Looks like I have to look at WildFly/EAP DMR to
see what is
>>>>>>> possible to
>>>>>>> > do with it, as I'm not sure if it is about
remote monitoring
>>>>>>> also and
>>>>>>> > if/how it can be use from monitoring systems
like Splunk.
>>>>>>> >
>>>>>>> > Vl.
>>>>>>> >
>>>>>>> > On 13.7.2015 15:26, Stian Thorgersen wrote:
>>>>>>> > > In WildFly/EAP that's DMR right?
We're planning to make
>>>>>>> Keycloak managable
>>>>>>> > > through that as well. For example
everything that goes
>>>>>>> > > into
>>>>>>> > > keycloak-server.json will eventually be
moved to
>>>>>>> standalone.xml. Same with
>>>>>>> > > admin endpoints, everything you can do
there you'll
>>>>>>> > > eventually
>>>>>>> be able to
>>>>>>> > > do through DMR and jboss-cli as well.
>>>>>>> > >
>>>>>>> > > However, IMO it would make sense to at
least expose
>>>>>>> > > Keycloak
>>>>>>> specific
>>>>>>> > > information through the admin endpoints and
console as
>>>>>>> > > well.
>>>>>>> Such number
>>>>>>> > > of sessions, etc..
>>>>>>> > >
>>>>>>> > > ----- Original Message -----
>>>>>>> > >> From: "Vlastimil Elias"
<velias(a)redhat.com
>>>>>>> <mailto:velias@redhat.com>>
>>>>>>> > >> To: keycloak-dev(a)lists.jboss.org
>>>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>>>> > >> Sent: Monday, 13 July, 2015 3:17:16 PM
>>>>>>> > >> Subject: [keycloak-dev] Operational
monitoring of
>>>>>>> > >> Keycloak
>>>>>>> > >> server
>>>>>>> > >>
>>>>>>> > >> Hi,
>>>>>>> > >>
>>>>>>> > >> as we deployed KC to production mode
for
>>>>>>> https://developers.redhat.com
>>>>>>> > >> we started to think about operational
monitoring, for
>>>>>>> > >> example
>>>>>>> from
>>>>>>> > >> Nagios or other systems of this type.
>>>>>>> > >>
>>>>>>> > >> KC user guide doesn't contain any
chapter covering this
>>>>>>> topic, also no
>>>>>>> > >> any success over google search, so
looks like KC doesn't
>>>>>>> > >> have
>>>>>>> > >> any
>>>>>>> > >> solution for this yet.
>>>>>>> > >> But I believe this is an important area
which must be
>>>>>>> > >> solved
>>>>>>> when KC is
>>>>>>> > >> used for production.
>>>>>>> > >>
>>>>>>> > >> I can imagine monitoring of JDBC
connection if JPA is
>>>>>>> > >> used,
>>>>>>> monitoring
>>>>>>> > >> of Mongo connection if used as store,
monitoring of LDAP
>>>>>>> connection if
>>>>>>> > >> LDAP federation is used etc.
>>>>>>> > >> Also some statistics like numbers of
active sso session,
>>>>>>> number of
>>>>>>> > >> logins per minute etc should be
provided there.
>>>>>>> > >>
>>>>>>> > >> Monitoring is not about Keycloak core
itself, it should
>>>>>>> > >> be
>>>>>>> available for
>>>>>>> > >> extension developers also. For example
we implemented own
>>>>>>> > >> UserFederationProvider which calls
backend REST services.
>>>>>>> > >> We should be able to add info about
this integration into
>>>>>>> monitoring
>>>>>>> > >> endpoint to be able to catch problems
with this REST API.
>>>>>>> > >>
>>>>>>> > >> It should be probably implemented same
way as used by
>>>>>>> > >> underlying
>>>>>>> > >> WildFly/EAP (JPA/JDBC is probably
available for
>>>>>>> > >> monitoring
>>>>>>> there). I'm
>>>>>>> > >> not sure if JMX is used there still or
if some new
>>>>>>> > >> framework
>>>>>>> > >> is
>>>>>>> > >> available for it.
>>>>>>> > >> Or KC should use some form of KC REST
API for this, which
>>>>>>> should be
>>>>>>> > >> extended by additional info from KC
extensions?
>>>>>>> > >>
>>>>>>> > >> What do you think?
>>>>>>> > >>
>>>>>>> > >> Vlastimil
>>>>>>> > >>
>>>>>>> > >> P.S we have
https://issues.jboss.org/browse/RHD-552 for
>>>>>>> > >> Red
>>>>>>> > >> Hat
>>>>>>> > >> Developer instance of KC
>>>>>>> > >>
>>>>>>> > >> --
>>>>>>> > >> Vlastimil Elias
>>>>>>> > >> Principal Software Engineer
>>>>>>> > >> jboss.org <http://jboss.org>
Development Team
>>>>>>> > >>
>>>>>>> > >>
_______________________________________________
>>>>>>> > >> keycloak-dev mailing list
>>>>>>> > >> keycloak-dev(a)lists.jboss.org
>>>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>>>> > >>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>> > >>
>>>>>>> >
>>>>>>> > --
>>>>>>> > Vlastimil Elias
>>>>>>> > Principal Software Engineer
>>>>>>> > jboss.org <http://jboss.org> Development
Team
>>>>>>> >
>>>>>>> >
>>>>>>> _______________________________________________
>>>>>>> keycloak-dev mailing list
>>>>>>> keycloak-dev(a)lists.jboss.org
>>>>>>> <mailto:keycloak-dev@lists.jboss.org>
>>>>>>>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
>>>>>>>
>>>>>> --
>>>>>> Vlastimil Elias
>>>>>> Principal Software Engineer
>>>>>> jboss.org Development Team
>>>>>>
>>>>>>
>>>> --
>>>> Vlastimil Elias
>>>> Principal Software Engineer
>>>> jboss.org Development Team
>>>>
>>>>
>> --
>> Vlastimil Elias
>> Principal Software Engineer
>> jboss.org Development Team
>>
>>
--
Vlastimil Elias
Principal Software Engineer
jboss.org Development Team
10:02 a.m.
Sorry for the late response.
Potentially, integration with Hawkular might be an option. http://www.hawkular.org/
----- Original Message -----
From: "Stian Thorgersen" <stian(a)redhat.com>
To: "Vlastimil Elias" <velias(a)redhat.com>
Cc: keycloak-dev(a)lists.jboss.org
Sent: Thursday, July 16, 2015 2:49:56 AM
Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
----- Original Message -----
> From: "Vlastimil Elias" <velias(a)redhat.com>
> To: "Stian Thorgersen" <stian(a)redhat.com>
> Cc: keycloak-dev(a)lists.jboss.org
> Sent: Thursday, 16 July, 2015 8:15:29 AM
> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>
> Hi,
>
> OK, if DB is "realm" and "users" then it have to be well
documented as
> it is not obvious for people without deeper knowledge of KC.
We could always call it realm-database or something, but we have 3 different
stores so should be separate status for them
>
> Do we have JIRA issue for this or should I create one?
Nope, so please create one
>
> Vl.
>
>
> On 14.7.2015 10:06, Stian Thorgersen wrote:
> >
> > ----- Original Message -----
> >> From: "Vlastimil Elias" <velias(a)redhat.com>
> >> To: "Stian Thorgersen" <stian(a)redhat.com>
> >> Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
> >> keycloak-dev(a)lists.jboss.org
> >> Sent: Tuesday, 14 July, 2015 9:58:36 AM
> >> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
> >>
> >> On 14.7.2015 09:14, Stian Thorgersen wrote:
> >>> ----- Original Message -----
> >>>> From: "Vlastimil Elias" <velias(a)redhat.com>
> >>>> To: "Stian Thorgersen" <stian(a)redhat.com>
> >>>> Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
> >>>> keycloak-dev(a)lists.jboss.org
> >>>> Sent: Tuesday, 14 July, 2015 8:56:43 AM
> >>>> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak
server
> >>>>
> >>>>
> >>>>
> >>>> On 14.7.2015 08:42, Stian Thorgersen wrote:
> >>>>> Ok so we add one that just returns OK or ERROR which
doesn't require
> >>>>> authentication, but then we add another that can return more
details,
> >>>>> including stats and what the error is.
> >>>> Yep. Simple http endpoinds for now, later all this info may be
expose
> >>>> over DMR or some other mechanism if necessary.
> >>>>> For full health we should check all external services that are
used
> >>>>> (db,
> >>>>> identity providers and user federation providers). Cluster sync
will
> >>>>> depend on what Infinispan exposes, but I imagine they'll
have the
> >>>>> required
> >>>>> details avail.
> >>>> Beside endpoint for full health check (which should summarize all
> >>>> services into one flag if I understand correctly, which is good
for
> >>>> loadbalancer) we should expose separate endpoint for each service
> >>>> health, as it is better for operational monitoring systems like
> >>>> Nagios.
> >>>> Admins are better informed what is going wrong and they can
resolve
> >>>> situation more quickly without necessity to search cause of
general
> >>>> failure flag somewhere else.
> >>> We could have a single health endpoint that returns a 200 or 500 and
> >>> also
> >>> returns a json doc with details. Something like:
> >>>
> >>> {
> >>> realm: ok,
> >>> users: ok,
> >>> realmCache: error,
> >>> userCache: error,
> >>> identity-providers: {
> >>> google: ok,
> >>> facebook: error
> >>> },
> >>> user-federation: {
> >>> ldap: ok
> >>> }
> >>> }
> >>>
> >>> I wonder if that's to much details to expose without
authentication.
> >> This should work, but I miss info about DB connection here. It is
> >> crucial from operational point of view as this allows sysadmins quickly
> >> detect that connection to DB is broken.
> >>
> >> I believe this kind of info can be safely exposed without
> >> authentication. If the endpoint is clearly documented then access can be
> >> always restricted on some network infra (webserver/proxy) ahead of KC.
> > DB is "realm" and "users".
> >
> >>>> Vl.
> >>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "Vlastimil Elias"
<velias(a)redhat.com>
> >>>>>> To: "Scott Rossillo"
<srossillo(a)smartling.com>, "Stian Thorgersen"
> >>>>>> <stian(a)redhat.com>
> >>>>>> Cc: keycloak-dev(a)lists.jboss.org
> >>>>>> Sent: Tuesday, 14 July, 2015 8:34:29 AM
> >>>>>> Subject: Re: [keycloak-dev] Operational monitoring of
Keycloak
> >>>>>> server
> >>>>>>
> >>>>>> Yap, beside server-stat endpoint Stian talked about we
should add
> >>>>>> some
> >>>>>> server-health endpoint also to be used for remote health
monitoring
> >>>>>> by
> >>>>>> load balancers or systems like Nagios.
> >>>>>> Simple http endpoint without authentication (as it
doesn't leak any
> >>>>>> sensitive information) is typically simplest way how to do
this
> >>>>>> monitoring.
> >>>>>>
> >>>>>> Vl.
> >>>>>>
> >>>>>> On 14.7.2015 05:13, Scott Rossillo wrote:
> >>>>>>> Some type of health page would be great too for load
balancers to
> >>>>>>> monitor. Something that doesn't leak internal
information but
> >>>>>>> checks
> >>>>>>> behind the scenes that:
> >>>>>>> 1. Server can reach its databas(es)
> >>>>>>> 2. Server cluster sync is working
> >>>>>>> 3. Server can reach federation providers, etc.
> >>>>>>> Endpoint should respond to get requests and return an
http status
> >>>>>>> reflective of server state.
> >>>>>>>
> >>>>>>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen
<stian(a)redhat.com
> >>>>>>> <mailto:stian@redhat.com>> wrote:
> >>>>>>>
> >>>>>>> So looks like we're at agreement to add the
additional info
> >>>>>>> you
> >>>>>>> wanted to server info page.
> >>>>>>>
> >>>>>>> How about we add an additional endpoint
server-stat that can
> >>>>>>> collect some stats about the server?
> >>>>>>>
> >>>>>>> ----- Original Message -----
> >>>>>>> > From: "Vlastimil Elias"
<velias(a)redhat.com
> >>>>>>> <mailto:velias@redhat.com>>
> >>>>>>> > To: "Stian Thorgersen"
<stian(a)redhat.com
> >>>>>>> > <mailto:stian@redhat.com>>
> >>>>>>> > Cc: keycloak-dev(a)lists.jboss.org
> >>>>>>> <mailto:keycloak-dev@lists.jboss.org>
> >>>>>>> > Sent: Monday, 13 July, 2015 5:06:34 PM
> >>>>>>> > Subject: Re: [keycloak-dev] Operational
monitoring of
> >>>>>>> > Keycloak
> >>>>>>> server
> >>>>>>> >
> >>>>>>> > Looks like I have to look at WildFly/EAP
DMR to see what
> >>>>>>> > is
> >>>>>>> possible to
> >>>>>>> > do with it, as I'm not sure if it is
about remote
> >>>>>>> > monitoring
> >>>>>>> also and
> >>>>>>> > if/how it can be use from monitoring
systems like Splunk.
> >>>>>>> >
> >>>>>>> > Vl.
> >>>>>>> >
> >>>>>>> > On 13.7.2015 15:26, Stian Thorgersen
wrote:
> >>>>>>> > > In WildFly/EAP that's DMR right?
We're planning to make
> >>>>>>> Keycloak managable
> >>>>>>> > > through that as well. For example
everything that goes
> >>>>>>> > > into
> >>>>>>> > > keycloak-server.json will eventually
be moved to
> >>>>>>> standalone.xml. Same with
> >>>>>>> > > admin endpoints, everything you can do
there you'll
> >>>>>>> > > eventually
> >>>>>>> be able to
> >>>>>>> > > do through DMR and jboss-cli as well.
> >>>>>>> > >
> >>>>>>> > > However, IMO it would make sense to at
least expose
> >>>>>>> > > Keycloak
> >>>>>>> specific
> >>>>>>> > > information through the admin
endpoints and console as
> >>>>>>> > > well.
> >>>>>>> Such number
> >>>>>>> > > of sessions, etc..
> >>>>>>> > >
> >>>>>>> > > ----- Original Message -----
> >>>>>>> > >> From: "Vlastimil Elias"
<velias(a)redhat.com
> >>>>>>> <mailto:velias@redhat.com>>
> >>>>>>> > >> To: keycloak-dev(a)lists.jboss.org
> >>>>>>> <mailto:keycloak-dev@lists.jboss.org>
> >>>>>>> > >> Sent: Monday, 13 July, 2015
3:17:16 PM
> >>>>>>> > >> Subject: [keycloak-dev]
Operational monitoring of
> >>>>>>> > >> Keycloak
> >>>>>>> > >> server
> >>>>>>> > >>
> >>>>>>> > >> Hi,
> >>>>>>> > >>
> >>>>>>> > >> as we deployed KC to production
mode for
> >>>>>>> https://developers.redhat.com
> >>>>>>> > >> we started to think about
operational monitoring, for
> >>>>>>> > >> example
> >>>>>>> from
> >>>>>>> > >> Nagios or other systems of this
type.
> >>>>>>> > >>
> >>>>>>> > >> KC user guide doesn't contain
any chapter covering this
> >>>>>>> topic, also no
> >>>>>>> > >> any success over google search, so
looks like KC
> >>>>>>> > >> doesn't
> >>>>>>> > >> have
> >>>>>>> > >> any
> >>>>>>> > >> solution for this yet.
> >>>>>>> > >> But I believe this is an important
area which must be
> >>>>>>> > >> solved
> >>>>>>> when KC is
> >>>>>>> > >> used for production.
> >>>>>>> > >>
> >>>>>>> > >> I can imagine monitoring of JDBC
connection if JPA is
> >>>>>>> > >> used,
> >>>>>>> monitoring
> >>>>>>> > >> of Mongo connection if used as
store, monitoring of
> >>>>>>> > >> LDAP
> >>>>>>> connection if
> >>>>>>> > >> LDAP federation is used etc.
> >>>>>>> > >> Also some statistics like numbers
of active sso
> >>>>>>> > >> session,
> >>>>>>> number of
> >>>>>>> > >> logins per minute etc should be
provided there.
> >>>>>>> > >>
> >>>>>>> > >> Monitoring is not about Keycloak
core itself, it should
> >>>>>>> > >> be
> >>>>>>> available for
> >>>>>>> > >> extension developers also. For
example we implemented
> >>>>>>> > >> own
> >>>>>>> > >> UserFederationProvider which calls
backend REST
> >>>>>>> > >> services.
> >>>>>>> > >> We should be able to add info
about this integration
> >>>>>>> > >> into
> >>>>>>> monitoring
> >>>>>>> > >> endpoint to be able to catch
problems with this REST
> >>>>>>> > >> API.
> >>>>>>> > >>
> >>>>>>> > >> It should be probably implemented
same way as used by
> >>>>>>> > >> underlying
> >>>>>>> > >> WildFly/EAP (JPA/JDBC is probably
available for
> >>>>>>> > >> monitoring
> >>>>>>> there). I'm
> >>>>>>> > >> not sure if JMX is used there
still or if some new
> >>>>>>> > >> framework
> >>>>>>> > >> is
> >>>>>>> > >> available for it.
> >>>>>>> > >> Or KC should use some form of KC
REST API for this,
> >>>>>>> > >> which
> >>>>>>> should be
> >>>>>>> > >> extended by additional info from
KC extensions?
> >>>>>>> > >>
> >>>>>>> > >> What do you think?
> >>>>>>> > >>
> >>>>>>> > >> Vlastimil
> >>>>>>> > >>
> >>>>>>> > >> P.S we have
https://issues.jboss.org/browse/RHD-552 for
> >>>>>>> > >> Red
> >>>>>>> > >> Hat
> >>>>>>> > >> Developer instance of KC
> >>>>>>> > >>
> >>>>>>> > >> --
> >>>>>>> > >> Vlastimil Elias
> >>>>>>> > >> Principal Software Engineer
> >>>>>>> > >> jboss.org <http://jboss.org>
Development Team
> >>>>>>> > >>
> >>>>>>> > >>
_______________________________________________
> >>>>>>> > >> keycloak-dev mailing list
> >>>>>>> > >> keycloak-dev(a)lists.jboss.org
> >>>>>>> <mailto:keycloak-dev@lists.jboss.org>
> >>>>>>> > >>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>>>>>> > >>
> >>>>>>> >
> >>>>>>> > --
> >>>>>>> > Vlastimil Elias
> >>>>>>> > Principal Software Engineer
> >>>>>>> > jboss.org <http://jboss.org>
Development Team
> >>>>>>> >
> >>>>>>> >
> >>>>>>> _______________________________________________
> >>>>>>> keycloak-dev mailing list
> >>>>>>> keycloak-dev(a)lists.jboss.org
> >>>>>>> <mailto:keycloak-dev@lists.jboss.org>
> >>>>>>>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
> >>>>>>>
> >>>>>> --
> >>>>>> Vlastimil Elias
> >>>>>> Principal Software Engineer
> >>>>>> jboss.org Development Team
> >>>>>>
> >>>>>>
> >>>> --
> >>>> Vlastimil Elias
> >>>> Principal Software Engineer
> >>>> jboss.org Development Team
> >>>>
> >>>>
> >> --
> >> Vlastimil Elias
> >> Principal Software Engineer
> >> jboss.org Development Team
> >>
> >>
>
> --
> Vlastimil Elias
> Principal Software Engineer
> jboss.org Development Team
>
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
2:57 a.m.
Not sure how Hawkular would help. All we want is some basic info about a single KC server,
or a cluster of KC servers. We're not monitoring multiple servers. I would more
imagine that what we produce would be something Hawkular would be able to consume rather
than us integrating/embedding Hawkular.
----- Original Message -----
From: "Brett Meyer" <brmeyer(a)redhat.com>
To: keycloak-dev(a)lists.jboss.org
Sent: Thursday, 13 August, 2015 5:02:58 PM
Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
Sorry for the late response.
Potentially, integration with Hawkular might be an option.
http://www.hawkular.org/
----- Original Message -----
> From: "Stian Thorgersen" <stian(a)redhat.com>
> To: "Vlastimil Elias" <velias(a)redhat.com>
> Cc: keycloak-dev(a)lists.jboss.org
> Sent: Thursday, July 16, 2015 2:49:56 AM
> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
>
>
>
> ----- Original Message -----
> > From: "Vlastimil Elias" <velias(a)redhat.com>
> > To: "Stian Thorgersen" <stian(a)redhat.com>
> > Cc: keycloak-dev(a)lists.jboss.org
> > Sent: Thursday, 16 July, 2015 8:15:29 AM
> > Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
> >
> > Hi,
> >
> > OK, if DB is "realm" and "users" then it have to be well
documented as
> > it is not obvious for people without deeper knowledge of KC.
>
> We could always call it realm-database or something, but we have 3
> different
> stores so should be separate status for them
>
> >
> > Do we have JIRA issue for this or should I create one?
>
> Nope, so please create one
>
> >
> > Vl.
> >
> >
> > On 14.7.2015 10:06, Stian Thorgersen wrote:
> > >
> > > ----- Original Message -----
> > >> From: "Vlastimil Elias" <velias(a)redhat.com>
> > >> To: "Stian Thorgersen" <stian(a)redhat.com>
> > >> Cc: "Scott Rossillo" <srossillo(a)smartling.com>,
> > >> keycloak-dev(a)lists.jboss.org
> > >> Sent: Tuesday, 14 July, 2015 9:58:36 AM
> > >> Subject: Re: [keycloak-dev] Operational monitoring of Keycloak server
> > >>
> > >> On 14.7.2015 09:14, Stian Thorgersen wrote:
> > >>> ----- Original Message -----
> > >>>> From: "Vlastimil Elias" <velias(a)redhat.com>
> > >>>> To: "Stian Thorgersen" <stian(a)redhat.com>
> > >>>> Cc: "Scott Rossillo"
<srossillo(a)smartling.com>,
> > >>>> keycloak-dev(a)lists.jboss.org
> > >>>> Sent: Tuesday, 14 July, 2015 8:56:43 AM
> > >>>> Subject: Re: [keycloak-dev] Operational monitoring of
Keycloak
> > >>>> server
> > >>>>
> > >>>>
> > >>>>
> > >>>> On 14.7.2015 08:42, Stian Thorgersen wrote:
> > >>>>> Ok so we add one that just returns OK or ERROR which
doesn't
> > >>>>> require
> > >>>>> authentication, but then we add another that can return
more
> > >>>>> details,
> > >>>>> including stats and what the error is.
> > >>>> Yep. Simple http endpoinds for now, later all this info may
be
> > >>>> expose
> > >>>> over DMR or some other mechanism if necessary.
> > >>>>> For full health we should check all external services that
are used
> > >>>>> (db,
> > >>>>> identity providers and user federation providers). Cluster
sync
> > >>>>> will
> > >>>>> depend on what Infinispan exposes, but I imagine
they'll have the
> > >>>>> required
> > >>>>> details avail.
> > >>>> Beside endpoint for full health check (which should summarize
all
> > >>>> services into one flag if I understand correctly, which is
good for
> > >>>> loadbalancer) we should expose separate endpoint for each
service
> > >>>> health, as it is better for operational monitoring systems
like
> > >>>> Nagios.
> > >>>> Admins are better informed what is going wrong and they can
resolve
> > >>>> situation more quickly without necessity to search cause of
general
> > >>>> failure flag somewhere else.
> > >>> We could have a single health endpoint that returns a 200 or 500
and
> > >>> also
> > >>> returns a json doc with details. Something like:
> > >>>
> > >>> {
> > >>> realm: ok,
> > >>> users: ok,
> > >>> realmCache: error,
> > >>> userCache: error,
> > >>> identity-providers: {
> > >>> google: ok,
> > >>> facebook: error
> > >>> },
> > >>> user-federation: {
> > >>> ldap: ok
> > >>> }
> > >>> }
> > >>>
> > >>> I wonder if that's to much details to expose without
authentication.
> > >> This should work, but I miss info about DB connection here. It is
> > >> crucial from operational point of view as this allows sysadmins
> > >> quickly
> > >> detect that connection to DB is broken.
> > >>
> > >> I believe this kind of info can be safely exposed without
> > >> authentication. If the endpoint is clearly documented then access can
> > >> be
> > >> always restricted on some network infra (webserver/proxy) ahead of
KC.
> > > DB is "realm" and "users".
> > >
> > >>>> Vl.
> > >>>>
> > >>>>> ----- Original Message -----
> > >>>>>> From: "Vlastimil Elias"
<velias(a)redhat.com>
> > >>>>>> To: "Scott Rossillo"
<srossillo(a)smartling.com>, "Stian Thorgersen"
> > >>>>>> <stian(a)redhat.com>
> > >>>>>> Cc: keycloak-dev(a)lists.jboss.org
> > >>>>>> Sent: Tuesday, 14 July, 2015 8:34:29 AM
> > >>>>>> Subject: Re: [keycloak-dev] Operational monitoring of
Keycloak
> > >>>>>> server
> > >>>>>>
> > >>>>>> Yap, beside server-stat endpoint Stian talked about we
should add
> > >>>>>> some
> > >>>>>> server-health endpoint also to be used for remote
health
> > >>>>>> monitoring
> > >>>>>> by
> > >>>>>> load balancers or systems like Nagios.
> > >>>>>> Simple http endpoint without authentication (as it
doesn't leak
> > >>>>>> any
> > >>>>>> sensitive information) is typically simplest way how
to do this
> > >>>>>> monitoring.
> > >>>>>>
> > >>>>>> Vl.
> > >>>>>>
> > >>>>>> On 14.7.2015 05:13, Scott Rossillo wrote:
> > >>>>>>> Some type of health page would be great too for
load balancers to
> > >>>>>>> monitor. Something that doesn't leak internal
information but
> > >>>>>>> checks
> > >>>>>>> behind the scenes that:
> > >>>>>>> 1. Server can reach its databas(es)
> > >>>>>>> 2. Server cluster sync is working
> > >>>>>>> 3. Server can reach federation providers, etc.
> > >>>>>>> Endpoint should respond to get requests and return
an http status
> > >>>>>>> reflective of server state.
> > >>>>>>>
> > >>>>>>> On Mon, Jul 13, 2015 at 11:18 AM Stian Thorgersen
> > >>>>>>> <stian(a)redhat.com
> > >>>>>>> <mailto:stian@redhat.com>> wrote:
> > >>>>>>>
> > >>>>>>> So looks like we're at agreement to add
the additional
> > >>>>>>> info
> > >>>>>>> you
> > >>>>>>> wanted to server info page.
> > >>>>>>>
> > >>>>>>> How about we add an additional endpoint
server-stat that
> > >>>>>>> can
> > >>>>>>> collect some stats about the server?
> > >>>>>>>
> > >>>>>>> ----- Original Message -----
> > >>>>>>> > From: "Vlastimil Elias"
<velias(a)redhat.com
> > >>>>>>> <mailto:velias@redhat.com>>
> > >>>>>>> > To: "Stian Thorgersen"
<stian(a)redhat.com
> > >>>>>>> > <mailto:stian@redhat.com>>
> > >>>>>>> > Cc: keycloak-dev(a)lists.jboss.org
> > >>>>>>>
<mailto:keycloak-dev@lists.jboss.org>
> > >>>>>>> > Sent: Monday, 13 July, 2015 5:06:34
PM
> > >>>>>>> > Subject: Re: [keycloak-dev]
Operational monitoring of
> > >>>>>>> > Keycloak
> > >>>>>>> server
> > >>>>>>> >
> > >>>>>>> > Looks like I have to look at
WildFly/EAP DMR to see what
> > >>>>>>> > is
> > >>>>>>> possible to
> > >>>>>>> > do with it, as I'm not sure if it
is about remote
> > >>>>>>> > monitoring
> > >>>>>>> also and
> > >>>>>>> > if/how it can be use from monitoring
systems like
> > >>>>>>> > Splunk.
> > >>>>>>> >
> > >>>>>>> > Vl.
> > >>>>>>> >
> > >>>>>>> > On 13.7.2015 15:26, Stian Thorgersen
wrote:
> > >>>>>>> > > In WildFly/EAP that's DMR
right? We're planning to
> > >>>>>>> > > make
> > >>>>>>> Keycloak managable
> > >>>>>>> > > through that as well. For example
everything that goes
> > >>>>>>> > > into
> > >>>>>>> > > keycloak-server.json will
eventually be moved to
> > >>>>>>> standalone.xml. Same with
> > >>>>>>> > > admin endpoints, everything you
can do there you'll
> > >>>>>>> > > eventually
> > >>>>>>> be able to
> > >>>>>>> > > do through DMR and jboss-cli as
well.
> > >>>>>>> > >
> > >>>>>>> > > However, IMO it would make sense
to at least expose
> > >>>>>>> > > Keycloak
> > >>>>>>> specific
> > >>>>>>> > > information through the admin
endpoints and console as
> > >>>>>>> > > well.
> > >>>>>>> Such number
> > >>>>>>> > > of sessions, etc..
> > >>>>>>> > >
> > >>>>>>> > > ----- Original Message -----
> > >>>>>>> > >> From: "Vlastimil
Elias" <velias(a)redhat.com
> > >>>>>>> <mailto:velias@redhat.com>>
> > >>>>>>> > >> To:
keycloak-dev(a)lists.jboss.org
> > >>>>>>>
<mailto:keycloak-dev@lists.jboss.org>
> > >>>>>>> > >> Sent: Monday, 13 July, 2015
3:17:16 PM
> > >>>>>>> > >> Subject: [keycloak-dev]
Operational monitoring of
> > >>>>>>> > >> Keycloak
> > >>>>>>> > >> server
> > >>>>>>> > >>
> > >>>>>>> > >> Hi,
> > >>>>>>> > >>
> > >>>>>>> > >> as we deployed KC to
production mode for
> > >>>>>>> https://developers.redhat.com
> > >>>>>>> > >> we started to think about
operational monitoring, for
> > >>>>>>> > >> example
> > >>>>>>> from
> > >>>>>>> > >> Nagios or other systems of
this type.
> > >>>>>>> > >>
> > >>>>>>> > >> KC user guide doesn't
contain any chapter covering
> > >>>>>>> > >> this
> > >>>>>>> topic, also no
> > >>>>>>> > >> any success over google
search, so looks like KC
> > >>>>>>> > >> doesn't
> > >>>>>>> > >> have
> > >>>>>>> > >> any
> > >>>>>>> > >> solution for this yet.
> > >>>>>>> > >> But I believe this is an
important area which must be
> > >>>>>>> > >> solved
> > >>>>>>> when KC is
> > >>>>>>> > >> used for production.
> > >>>>>>> > >>
> > >>>>>>> > >> I can imagine monitoring of
JDBC connection if JPA is
> > >>>>>>> > >> used,
> > >>>>>>> monitoring
> > >>>>>>> > >> of Mongo connection if used
as store, monitoring of
> > >>>>>>> > >> LDAP
> > >>>>>>> connection if
> > >>>>>>> > >> LDAP federation is used etc.
> > >>>>>>> > >> Also some statistics like
numbers of active sso
> > >>>>>>> > >> session,
> > >>>>>>> number of
> > >>>>>>> > >> logins per minute etc should
be provided there.
> > >>>>>>> > >>
> > >>>>>>> > >> Monitoring is not about
Keycloak core itself, it
> > >>>>>>> > >> should
> > >>>>>>> > >> be
> > >>>>>>> available for
> > >>>>>>> > >> extension developers also.
For example we implemented
> > >>>>>>> > >> own
> > >>>>>>> > >> UserFederationProvider which
calls backend REST
> > >>>>>>> > >> services.
> > >>>>>>> > >> We should be able to add info
about this integration
> > >>>>>>> > >> into
> > >>>>>>> monitoring
> > >>>>>>> > >> endpoint to be able to catch
problems with this REST
> > >>>>>>> > >> API.
> > >>>>>>> > >>
> > >>>>>>> > >> It should be probably
implemented same way as used by
> > >>>>>>> > >> underlying
> > >>>>>>> > >> WildFly/EAP (JPA/JDBC is
probably available for
> > >>>>>>> > >> monitoring
> > >>>>>>> there). I'm
> > >>>>>>> > >> not sure if JMX is used there
still or if some new
> > >>>>>>> > >> framework
> > >>>>>>> > >> is
> > >>>>>>> > >> available for it.
> > >>>>>>> > >> Or KC should use some form of
KC REST API for this,
> > >>>>>>> > >> which
> > >>>>>>> should be
> > >>>>>>> > >> extended by additional info
from KC extensions?
> > >>>>>>> > >>
> > >>>>>>> > >> What do you think?
> > >>>>>>> > >>
> > >>>>>>> > >> Vlastimil
> > >>>>>>> > >>
> > >>>>>>> > >> P.S we have
https://issues.jboss.org/browse/RHD-552
> > >>>>>>> > >> for
> > >>>>>>> > >> Red
> > >>>>>>> > >> Hat
> > >>>>>>> > >> Developer instance of KC
> > >>>>>>> > >>
> > >>>>>>> > >> --
> > >>>>>>> > >> Vlastimil Elias
> > >>>>>>> > >> Principal Software Engineer
> > >>>>>>> > >> jboss.org
<http://jboss.org> Development Team
> > >>>>>>> > >>
> > >>>>>>> > >>
_______________________________________________
> > >>>>>>> > >> keycloak-dev mailing list
> > >>>>>>> > >> keycloak-dev(a)lists.jboss.org
> > >>>>>>>
<mailto:keycloak-dev@lists.jboss.org>
> > >>>>>>> > >>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > >>>>>>> > >>
> > >>>>>>> >
> > >>>>>>> > --
> > >>>>>>> > Vlastimil Elias
> > >>>>>>> > Principal Software Engineer
> > >>>>>>> > jboss.org <http://jboss.org>
Development Team
> > >>>>>>> >
> > >>>>>>> >
> > >>>>>>>
_______________________________________________
> > >>>>>>> keycloak-dev mailing list
> > >>>>>>> keycloak-dev(a)lists.jboss.org
> > >>>>>>>
<mailto:keycloak-dev@lists.jboss.org>
> > >>>>>>>
https://lists.jboss.org/mailman/listinfo/keycloak-dev
> > >>>>>>>
> > >>>>>> --
> > >>>>>> Vlastimil Elias
> > >>>>>> Principal Software Engineer
> > >>>>>> jboss.org Development Team
> > >>>>>>
> > >>>>>>
> > >>>> --
> > >>>> Vlastimil Elias
> > >>>> Principal Software Engineer
> > >>>> jboss.org Development Team
> > >>>>
> > >>>>
> > >> --
> > >> Vlastimil Elias
> > >> Principal Software Engineer
> > >> jboss.org Development Team
> > >>
> > >>
> >
> > --
> > Vlastimil Elias
> > Principal Software Engineer
> > jboss.org Development Team
> >
> >
> _______________________________________________
> keycloak-dev mailing list
> keycloak-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-dev
>
_______________________________________________
keycloak-dev mailing list
keycloak-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/keycloak-dev
3033
days inactive
3068
days old
14 comments
4 participants
participants (4)
-
Brett Meyer -
Scott Rossillo -
Stian Thorgersen -
Vlastimil Elias