[Hawkular-dev] Sync with Inventory
Austin Kuo
auszon3 at gmail.com
Fri Jul 8 10:05:43 EDT 2016
Ok, that way, we can just use '/sync' to sycn with inventory.
On Fri, Jul 8, 2016 at 10:00 PM Thomas Segismont <tsegismo at redhat.com>
wrote:
> Hi Austin,
>
> Not sure what you meant, but my opinion on the problem so far is that we
> should have a feed per Vert.x instance. Eventually (but that is beyond
> the scope of your project) we could find out when two Vert.x feeds are
> using the same clustered EventBus and then create a "logical" resource
> in inventory to represent it.
>
> Thomas
>
> Le 05/07/2016 à 01:53, Austin Kuo a écrit :
> > why "isParentOf" is more suitable than "contains" in vertx as Thomas
> said?
> >
> > If it is "contains", it also makes sense to me since if "MyApp" is gone,
> > the feeds it contains should disappear as well.
> >
> > Austin
> > Lukas Krejci <lkrejci at redhat.com <mailto:lkrejci at redhat.com>>於 2016年6
> > 月29日 週三，21:20寫道：
> >
> > Btw. I've slightly updated the inventory organization description on
> the
> > hawkular site (http://www.hawkular.org/docs/components/inventory/
> > index.html#inventory-organization
> > <
> http://www.hawkular.org/docs/components/inventory/index.html#inventory-organization
> >).
> > I hope it explains the structure and
> > intent of the entities in inventory in a slightly more
> > comprehensible manner.
> >
> > My answers are inline below...
> >
> > On středa 29. června 2016 14:39:27 CEST Thomas Segismont wrote:
> > > Thank you very much for the thorough reply Lukas. A few
> > > questions/comments inline.
> > >
> > > Le 23/06/2016 à 15:59, Lukas Krejci a écrit :
> > > > On Thursday, June 23, 2016 10:27:12 AM Thomas Segismont wrote:
> > > >> Hey Lukas,
> > > >>
> > > >> Thank you for pointing us in the sync endpoint. Austin will
> > look into
> > > >> this and will certainly come back with more questions.
> > > >>
> > > >> With respect to the user creating resources question, the
> > difference
> > > >> between Vert.x and Wildfly is that the user creates resources
> > > >> grammatically. So in version 1 of the application, there might
> > be two
> > > >> HTTP servers as well as 7 event bus handlers, but only 1 http
> > server in
> > > >> version 2. And a named worker pool in version 3.
> > > >>
> > > >> In the end, I believe it doesn't matter if it's container which
> > creates
> > > >> resources or if it's the user himself. Does it?
> > > >
> > > > It does not really (inventory has just a single API, so it does
> > not really
> > > > know who is talking to it - if a feed or if a user) - but
> > resources inside
> > > > and outside feeds have slightly different semantics.
> > > >
> > > > Right now the logic is this:
> > > >
> > > > Feeds are "agents" that don't care about anything else but their
> own
> > > > little
> > > > "world". That's why they can create their own resource types,
> > metric types
> > > > and they also declare resources and metrics of those types. Feed
> > does not
> > > > need to look "outside" of its own data and is in full charge of
> it.
> > >
> > > Does that mean that creating a feed is the only way to create
> > > resource/metric types?
> >
> > No, you can also create resource types and metric types directly
> > under the
> > tenant.
> >
> > > I suppose the benefit of creating resource types is that then you
> can
> > > search for different resources of the same type easily.
> > >
> > > And if feeds create resource types, how do you know that resource
> > types
> > > created by the Hawkular Agent feed running on server A are the
> same as
> > > those created by another agent running on server B?
> > >
> >
> > Inventory automatically computes "identity hashes" of resource types
> and
> > metric types - if 2 resource types in 2 feeds have the same ID and
> > exactly the
> > same configuration definitions, they are considered identical. If
> > you know 1
> > resource type, you can find all the identical ones using the
> > following REST
> > API (since 0.17.0.Final, the format of the URLs is thoroughly
> > explained here:
> > http://www.hawkular.org/docs/rest/rest-inventory.html#_api_endpoints
> ):
> >
> > /hawkular/inventory/traversal/f;feedId/rt;resourceTypeId/identical
> >
> > If for example some resource types should be known up-front and
> "shared"
> > across all feeds, some kind of "gluecode" could create "global"
> > resource types
> > under the tenant, that would have the same id and structure as the
> > types that
> > the feeds declare. If then you want to for example find all
> > resources of given
> > type, you can:
> >
> >
> /hawkular/inventory/traversal/rt;myType/identical/rl;defines/type=resource
> >
> > I.e. for all types identical to the global one, find all resources
> > defined by
> > those types.
> >
> > > > Hence the /sync endpoint applies to a feed nicely - since it is
> > in charge,
> > > > it merely declares what is the view it has currently of the
> > "world" it
> > > > sees and inventory will make sure it has the same picture -
> > under that
> > > > feed.
> > > >
> > > > Now if you have an application that spans multiple vms/machines
> > and is
> > > > composed of multiple processes, there is no such clear
> > distinction of
> > > > "ownership".
> > >
> > > Good point, Vert.x applications are often distributed and
> > communicating
> > > over the EventBus.
> > >
> > > > While indeed a "real" user can just act like a feed, the
> envisioned
> > > > workflow is that the user operates directly in environments and
> > at the
> > > > top level. I.e. a user assigns feeds to environments (i.e. this
> feed
> > > > reports on my server in staging environment, etc) and the user
> > creates
> > > > "logical" resources in the environment (i.e. "My App" resource
> > in staging
> > > > env is composed of a load balancer managed by this feed, mongodb
> > managed
> > > > by another feed there and clustered wflys there, there and
> there).
> > > >
> > > > To model this, inventory supports 2 kinds of tree hierarchies -
> > 1 created
> > > > using the "contains" relationship, which expresses existential
> > ownership -
> > > > i.e. a feed contains its resources and if a feed disappears, so
> > do the
> > > > resources, because no one else can report on them. The entities
> > bound by
> > > > the
> > > How does a feed "disappear"? That would be by deleting it through
> the
> > > REST API, correct? Something the ManageIQ provider would do
> > through the
> > > Ruby client?
> > >
> >
> > yes
> >
> > > > contains relationship form a tree - no loops or diamonds in it
> > (this is
> > > > enforced by inventory). But there can also be a hierarchy
> > created using an
> > > > "isParentOf" relationship (which represents "logical" ownership).
> > > > Resources
> > > > bound by "isParentOf" can form an acyclic graph - i.e. 1
> > resource can have
> > > > multiple parents as well as many children (isParentOf is
> > applicable only
> > > > to
> > > > resources, not other types of entities).
> > > >
> > > > The hierarchies formed by "contains" and "isParentOf" are
> > independent. So
> > > > you can create a resource "My App" in the staging environment
> > and declare
> > > > it a parent (using "isParentOf") of the resources declared by
> > feeds that
> > > > manage the machines where the constituent servers live.
> > >
> > > Interesting, that may be the way to model a Vert.x app deployed on
> two
> > > machines. Each process would have its own feed reporting discovered
> > > resources (http servers, event bus handlers, ... etc), and a
> > logical app
> > > resource as parent.
> > >
> >
> > Exactly.
> >
> > > > That is the envisaged workflow for "apps". Now the downside to
> > that is
> > > > that
> > > > (currently) there is no "sync" for that. The reason is that the
> > > > application
> > > > really is a logical concept and the underlying servers can be
> > repurposed
> > > > to
> > > > serve different applications (so if app stops using it, it
> shouldn't
> > > > really
> > > > disappear from inventory, as is the case with /sync - because if
> > a feed
> > > > doesn't "see" a resource, then it really is just gone, because
> > the feed is
> > > > solely responsible for reporting on it).
> > >
> > > What happens to the resources exactly? Are they marked as gone or
> > simply
> > > deleted?
> >
> > Right now they are deleted. That is of course not optimal and
> > versioning is in
> > the pipeline right after the port of inventory to Tinkerpop3.
> > Basically all
> > the entities and relationships will get "from" and "to" timestamps.
> > Implicitly, you'd look at the "present", but you'd be able to look
> > at how
> > things looked in the past by specifying a different "now" in your
> query.
> >
> > > Do you know how dependent services are updated? For example, when
> > a JMS
> > > queue is gone, are alert definitions on queue depth removed as
> > well? How
> > > does that happen?
> > >
> >
> > Inventory sends events on the bus about every C/U/D of every entity
> or
> > relationship, so other components can react on that.
> >
> > > > We can think about how to somehow help clients with "App sync"
> > but I'm not
> > > > sure if having a feed for vertx is the right thing to do. On the
> > other
> > > > hand I very well may not be seeing some obvious problems of the
> > above or
> > > > parallels that make the 2 approaches really the same because the
> > above
> > > > model is just ingrained in my brain after so many hours thinking
> > about it
> > > > ;)
> > > >
> > > >> As for the feed question, the Vert.x feed will be the Metrics
> SPI
> > > >> implementation (vertx-hawkular-metrics project). Again I guess
> > it's not
> > > >> much different than the Hawkular Agent.
> > > >
> > > > A feed would only be appropriate if vertx app never reported on
> > something
> > > > that would also be reported by other agents. I.e. if a part of a
> > vertx
> > > > application is also reported on by a wfly agent, because that
> > part is
> > > > running in a wfly server managed by us, then that will not work
> - 1
> > > > resource cannot be "contained" in 2 different feeds (not just
> > API wise,
> > > > but logically, too).
> > > I'm not too worried about this use case. First the vast majority of
> > > Vert.x applications I know about are not embedded. Secondly the
> Vert.x
> > > feed would not report resources already reported by the Hawkular
> > Agent.
> > >
> > > >> Maybe the wording around user creating resources was confusing?
> > Did you
> > > >> thought he would do so from application code? In this case, the
> > answer
> > > >> is no.
> > > >
> > > > Yeah, we should probably get together and discuss what your
> > plans are to
> > > > get on the same page with everything.
> > >
> > > I believe that presenting to you (and to whoever is interested) the
> > > conclusions of investigations would be beneficial indeed.
> > >
> >
> > +1
> >
> > > >> Regards,
> > > >> Thomas
> > > >>
> > > >> Le 23/06/2016 à 10:01, Austin Kuo a écrit :
> > > >>> Yes, I’m gonna build the inventory for vertx applications.
> > > >>> So I have to create a feed for it.
> > > >>>
> > > >>> Thanks!
> > > >>>
> > > >>> On Tue, Jun 21, 2016 at 7:55 PM Lukas Krejci
> > <lkrejci at redhat.com <mailto:lkrejci at redhat.com>
> > > >>>
> > > >>> <mailto:lkrejci at redhat.com <mailto:lkrejci at redhat.com>>>
> wrote:
> > > >>> Hi Austin,
> > > >>>
> > > >>> Inventory offers a /hawkular/inventory/sync endpoint that
> > is used to
> > > >>> synchronize the "world view" of feeds (feed being
> > something that
> > > >>> pushes data
> > > >>> into inventory).
> > > >>>
> > > >>> You said though that a "user creates" the resources, so I
> > am not
> > > >>> sure if /sync
> > > >>> would be applicable to your scenario. Would you please
> > elaborate
> > > >>> more on where
> > > >>> in the inventory hierarchy you create your resources and
> > how? I.e.
> > > >>> are you
> > > >>> using some sort of feed akin to Hawkular's Wildfly Agent
> > or are you
> > > >>> just
> > > >>> creating your resources "manually" under environments?
> > > >>>
> > > >>> On Tuesday, June 21, 2016 02:20:33 AM Austin Kuo wrote:
> > > >>> > Hi all,
> > > >>> >
> > > >>> > I’m currently investigating how to sync with inventory
> > server.
> > > >>> > Here’s the example scenario:
> > > >>> > Consider the following problem. A user creates version 1
> > of the
> > > >>>
> > > >>> app with
> > > >>>
> > > >>> > two http servers, one listening on port 8080, the other
> > on port
> > > >>>
> > > >>> 8181. In
> > > >>>
> > > >>> > version 2, the http server listening on port 8181 is no
> > longer
> > > >>> > needed.
> > > >>> > When the old version is stopped and the new version
> > started, there
> > > >>>
> > > >>> will be
> > > >>>
> > > >>> > just one http server listening. The application is not
> > aware of
> > > >>> > the
> > > >>> > previous state. What should we do so that the second
> > http server
> > > >>>
> > > >>> is removed
> > > >>>
> > > >>> > from Inventory?
> > > >>> >
> > > >>> > Thanks in advance.
> > > >>>
> > > >>> --
> > > >>> Lukas Krejci
> > > >>>
> > > >>> _______________________________________________
> > > >>> hawkular-dev mailing list
> > > >>> hawkular-dev at lists.jboss.org
> > <mailto:hawkular-dev at lists.jboss.org>
> > <mailto:hawkular-dev at lists.jboss.org
> > <mailto:hawkular-dev at lists.jboss.org>>
> > > >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > > >>>
> > > >>> _______________________________________________
> > > >>> hawkular-dev mailing list
> > > >>> hawkular-dev at lists.jboss.org <mailto:
> hawkular-dev at lists.jboss.org>
> > > >>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> >
> > --
> > Lukas Krejci
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org <mailto:hawkular-dev at lists.jboss.org>
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> >
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
>
> --
> Thomas Segismont
> JBoss ON Engineering Team
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hawkular-dev/attachments/20160708/5b5669e6/attachment-0001.html
More information about the hawkular-dev
mailing list