[Hawkular-dev] agent "new server" event

[John Mazzitelli]

> The idea to limit inventory's responsibilities, and then the metrics TTL
> discussion, sort of led me towards another idea.  Perhaps we could
> eliminate inventory and metrics from this feature, and also let the
> agent be "dumb" and just configuration driven.  We could add agent
> meta-data on resource types we want reported when newly-discovered.  I
> suppose we could have a variety of notification-types but
> "on-initial-discovery" would be the only thing of interest at the
> moment, and only on server types.
> 
> The agent could make a rest call to hServices (glue code) notifying of a
> new resource discovery.  It could report this multiple times (like on
> every startup).  The glue code would just check to see if there was
> already an event for this and if not it would generate one.  This is via
> the alerting API, bypassing metrics and inventory, no TTL issues, etc.
> The logic is centralized and the agent doesn't really have to do
> anything very smart (I think).  We could feasibly do other things in the
> hServices code, like update an existing event (via tag) each time to
> keep track of recency, perhaps allowing some sort of event generation
> for "removed resource" when we haven't been updated in a long time.
> 
> Thoughts?

To expand on the metadata idea - assuming we may want something else like this in the future (maybe, "resource removed"??) we could have something on resource type like "server-notification" and its a list of notifications we want the agent to send up for resources of that type.

Say:

resource-type-dmr:
  - name: My Server
  - server-notifications:
    - name: resource-added
    - name: resource-removed

So when a resource of that type is added, the agent will send some "resource added" event to the server. If a resource of that type is removed the agent will send some "resource removed" event to the server (which we won't implement yet, but that's the idea how we can add events in the future if we need them). The names of the notifications are a fixed set. The agent would need to document the notification names it supports. We would only support one to start - "resource-added" - for this feature we are talking about in this thread.

The glue code can then take over and do what needs to be done for the messages the agent sends.

We would need at least one REST endpoint in the server. One per notification type, I guess.

Note that this won't work in "Metrics Only" mode and would be disabled since the glue code only exists in "hawkular full mode".