[Hawkular-dev] Sync with Inventory

Thu Jun 23 09:59:37 EDT 2016

On Thursday, June 23, 2016 10:27:12 AM Thomas Segismont wrote:
> Hey Lukas,
> 
> Thank you for pointing us in the sync endpoint. Austin will look into
> this and will certainly come back with more questions.
> 
> With respect to the user creating resources question, the difference
> between Vert.x and Wildfly is that the user creates resources
> grammatically. So in version 1 of the application, there might be two
> HTTP servers as well as 7 event bus handlers, but only 1 http server in
> version 2. And a named worker pool in version 3.
> 
> In the end, I believe it doesn't matter if it's container which creates
> resources or if it's the user himself. Does it?
> 

It does not really (inventory has just a single API, so it does not really 
know who is talking to it - if a feed or if a user) - but resources inside and 
outside feeds have slightly different semantics.

Right now the logic is this:

Feeds are "agents" that don't care about anything else but their own little 
"world". That's why they can create their own resource types, metric types and 
they also declare resources and metrics of those types. Feed does not need to 
look "outside" of its own data and is in full charge of it.

Hence the /sync endpoint applies to a feed nicely - since it is in charge, it 
merely declares what is the view it has currently of the "world" it sees and 
inventory will make sure it has the same picture - under that feed.

Now if you have an application that spans multiple vms/machines and is 
composed of multiple processes, there is no such clear distinction of 
"ownership".

While indeed a "real" user can just act like a feed, the envisioned workflow 
is that the user operates directly in environments and at the top level. I.e. 
a user assigns feeds to environments (i.e. this feed reports on my server in 
staging environment, etc) and the user creates "logical" resources in the 
environment (i.e. "My App" resource in staging env is composed of a load 
balancer managed by this feed, mongodb managed by another feed there and 
clustered wflys there, there and there). 

To model this, inventory supports 2 kinds of tree hierarchies - 1 created 
using the "contains" relationship, which expresses existential ownership - 
i.e. a feed contains its resources and if a feed disappears, so do the 
resources, because no one else can report on them. The entities bound by the 
contains relationship form a tree - no loops or diamonds in it (this is 
enforced by inventory). But there can also be a hierarchy created using an 
"isParentOf" relationship (which represents "logical" ownership). Resources 
bound by "isParentOf" can form an acyclic graph - i.e. 1 resource can have 
multiple parents as well as many children (isParentOf is applicable only to 
resources, not other types of entities).

The hierarchies formed by "contains" and "isParentOf" are independent. So you 
can create a resource "My App" in the staging environment and declare it a 
parent (using "isParentOf") of the resources declared by feeds that manage the 
machines where the constituent servers live.

That is the envisaged workflow for "apps". Now the downside to that is that 
(currently) there is no "sync" for that. The reason is that the application 
really is a logical concept and the underlying servers can be repurposed to 
serve different applications (so if app stops using it, it shouldn't really 
disappear from inventory, as is the case with /sync - because if a feed 
doesn't "see" a resource, then it really is just gone, because the feed is 
solely responsible for reporting on it).

We can think about how to somehow help clients with "App sync" but I'm not 
sure if having a feed for vertx is the right thing to do. On the other hand I 
very well may not be seeing some obvious problems of the above or parallels 
that make the 2 approaches really the same because the above model is just 
ingrained in my brain after so many hours thinking about it ;)

> As for the feed question, the Vert.x feed will be the Metrics SPI
> implementation (vertx-hawkular-metrics project). Again I guess it's not
> much different than the Hawkular Agent.
> 

A feed would only be appropriate if vertx app never reported on something that 
would also be reported by other agents. I.e. if a part of a vertx application 
is also reported on by a wfly agent, because that part is running in a wfly 
server managed by us, then that will not work - 1 resource cannot be 
"contained" in 2 different feeds (not just API wise, but logically, too).

> Maybe the wording around user creating resources was confusing? Did you
> thought he would do so from application code? In this case, the answer
> is no.
> 

Yeah, we should probably get together and discuss what your plans are to get 
on the same page with everything.

> Regards,
> Thomas
> 
> Le 23/06/2016 à 10:01, Austin Kuo a écrit :
> > Yes, I’m gonna build the inventory for vertx applications.
> > So I have to create a feed for it.
> > 
> > Thanks!
> > 
> > On Tue, Jun 21, 2016 at 7:55 PM Lukas Krejci <lkrejci at redhat.com
> > 
> > <mailto:lkrejci at redhat.com>> wrote:
> >     Hi Austin,
> >     
> >     Inventory offers a /hawkular/inventory/sync endpoint that is used to
> >     synchronize the "world view" of feeds (feed being something that
> >     pushes data
> >     into inventory).
> >     
> >     You said though that a "user creates" the resources, so I am not
> >     sure if /sync
> >     would be applicable to your scenario. Would you please elaborate
> >     more on where
> >     in the inventory hierarchy you create your resources and how? I.e.
> >     are you
> >     using some sort of feed akin to Hawkular's Wildfly Agent or are you
> >     just
> >     creating your resources "manually" under environments?
> >     
> >     On Tuesday, June 21, 2016 02:20:33 AM Austin Kuo wrote:
> >     > Hi all,
> >     > 
> >     > I’m currently investigating how to sync with inventory server.
> >     > Here’s the example scenario:
> >     > Consider the following problem. A user creates version 1 of the
> >     
> >     app with
> >     
> >     > two http servers, one listening on port 8080, the other on port
> >     
> >     8181. In
> >     
> >     > version 2, the http server listening on port 8181 is no longer
> >     > needed.
> >     > When the old version is stopped and the new version started, there
> >     
> >     will be
> >     
> >     > just one http server listening. The application is not aware of the
> >     > previous state. What should we do so that the second http server
> >     
> >     is removed
> >     
> >     > from Inventory?
> >     > 
> >     > Thanks in advance.
> >     
> >     --
> >     Lukas Krejci
> >     
> >     _______________________________________________
> >     hawkular-dev mailing list
> >     hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
> >     https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > 
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev

-- 
Lukas Krejci