Re: [infinispan-dev] Health check use cases

Hey Tristan! Comments inlined. Thanks Sebastian On Tue, Jul 26, 2016 at 9:50 AM, Tristan Tarrant <ttarrant(a)redhat.com <mailto:ttarrant@redhat.com>> wrote: On 26/07/16 07:10, Sebastian Laskawiec wrote: > Dear Community, > > I'd like to ask you for help. I'm currently sketching a design for a > REST health check endpoint for Infinispan and I'm trying to imagine > possible use cases. The health-check should be implemented as an MBean initially, with the ability to expose it via alternative implementations later. The server RESTful endpoint should be registered with the management interface via a special handler. Yes, I think it's a good idea. We could even use tools like Jolokia [1] to expose MBeans through REST interface (it can be added to standalone.conf to the bootstrap classpath). Alternatively we could use JDK embedded HTTP Server [2].

A cache and cachemanager's health is determined by a combination of parameters and we probably should allow for a user-pluggable checker. We already expose a number of statuses already, although obviously this would be an aggregate. Could you please elaborate more on that? How do we expose this information? Are you referring to Infinispan Stats [3]? I also though about supporting queries somehow. An imaginary example from the top of my head could look like the following: http://$ISPN/_health?cluster.nodes=3&MyCacheManager.MyCache.status=DEGRADED //<-- This would return 200 OK if we have 3 nodes and given cache is in degraded mode. http://$ISPN/_health?cluster.nodes=3&MyCacheManager.rebalance=IN_PROGRESS //<-- Checks if we have 3 nodes and rebalance is in proress [3] https://github.com/infinispan/infinispan/tree/8.2.x/core/src/main/java/or... > > Could you please give me a hand and tell me what functionalities are > important for you? Would you like to be able to check status per-cache > or maybe a red (not healthy), green (healthy), yellow (healthy, > rebalance in progress) cluster status is sufficient? What kind of > information do you expect to be there? I wouldn't want this to be overly complex: a simple OK, KO should be sufficient. Additional detail may be optionally present, but not a requirement. I think we will need at least a 3rd state - yellow or something like this. This would mean that a rebalance is in progress of a node is joining/leaving. In other words - the cluster accepts requests but don't touch the nodes!