[Hawkular-dev] metrics on the bus
Lucas Ponce
lponce at redhat.com
Wed Aug 10 03:37:07 EDT 2016
Today we have a business issue that we need to address:
https://issues.jboss.org/browse/HAWKULAR-1102
After reading the thread (and clean the expressions "piss on", "sane" and others) we can extract two main ideas:
- Re-design the architecture, questioning coupling of components and roles of existing pieces like the use of the messaging infrastructure.
- Proposing changes whithin the current architecture.
I don't see bad/good both approaches, but first one is something more for long term meanwhile the second one has a short term context.
Patterns are guides and they have scenarios where they could suit naturally and others where it is artificial and probably it won't help.
In our present design, we have a producer/consumer design using a messaging infrastructure, what we are proposing is that the producer won't send everything to the messaging as a short term workaround.
This is a perfectly valid like others and it has a degree of style which solution to use.
Also in terms of effort it could be worth to put in practice as it has less cost rather that a main architecture change as it is proposed as first option.
As this idea has an explicit veto to put it in practice, how do you want to proceed ? Polling or someone has the right to decide which decision/action to take ?
----- Mensaje original -----
> De: "John Mazzitelli" <mazz at redhat.com>
> Para: "Discussions around Hawkular development" <hawkular-dev at lists.jboss.org>
> Enviados: Martes, 9 de Agosto 2016 19:39:02
> Asunto: Re: [Hawkular-dev] metrics on the bus
>
> just for the record, we can't remove hawkular-bus - we use it for other
> things than just metrics :)
>
> ----- Original Message -----
> > Hi,
> >
> > Every inserted metric has to be processed inside metrics anyway, so it's an
> > event for us. And we can't tell if some system wants it. The filtering has
> > to be done somewhere and it's certainly not the metrics' job and event bus
> > shouldn't work in that way either. Event bus should be done the one doing
> > the filtering and routing, not the components producing the data. Why would
> > we do the things backwards? We should remove the whole metrics-bus
> > component
> > if we do something on the metrics side - it has no purpose anymore in any
> > sane architectural view.
> >
> > There's no reason to push this work to metrics without removing
> > hawkular-bus.
> > It makes more sense to integrate alerts to metrics directly then - and I
> > thought we wanted separated components.
> >
> > Is there a reason we can't use known-to-work-architecture?
> >
> > https://github.com/google/guava/wiki/EventBusExplained
> >
> > If we add the functionality to metrics, it should happen with this (as we
> > already have Guava and vertx isn't an option). And then it should remove
> > hawkular-bus as it's been replaced.
> >
> > - Micke
> >
> > ----- Original Message -----
> > From: "Jay Shaughnessy" <jshaughn at redhat.com>
> > To: hawkular-dev at lists.jboss.org
> > Sent: Tuesday, August 9, 2016 4:12:24 PM
> > Subject: Re: [Hawkular-dev] metrics on the bus
> >
> >
> > An event bus is for publishing events and I'm not sure every datum is an
> > event of any significance. The storage of an interesting datum could be an
> > event, and what is interesting can't be determined by the storage mechanism
> > unless told. It's not unlike adding a trigger to a db, or registering a
> > listener to be informed of significant happenings among a larger set of
> > happenings. So, to a degree I agree with Micke, publishing all events to
> > the
> > bus is good. But that leaves the semantic of choosing what an event is, and
> > I'd say every datum is not an event. And I'd also say that just because a
> > bus can handle millions of events, there is a practicality to not doing a
> > bunch of unnecessary work, and instead leaving that bandwidth available to
> > be used for solving actual business problems.
> >
> > I agree with Heiko that there are drawbacks to the polling approach (as
> > usual). But it has the advantage of not needing any further work on the
> > metrics side. But since Metrics already has hooks for bus publishing in
> > hawkular, perhaps it wouldn't be hard to add some selection as well. I'm
> > also fine with some sort of listener approach that leverages the
> > co-location
> > of components. Maybe some sort of Observer. Perhaps Micke or other metrics
> > guys could comment on that possibility.
> >
> >
> > On 8/9/2016 6:18 AM, Michael Burman wrote:
> >
> >
> >
> > Hi,
> >
> > Like I said in the IRC, I don't like either approach and both sort of piss
> > on
> > the well known EIP pattern. Since this problem has been solved ages ago,
> > why
> > are we going to reinvent it? Use a bus, and I mean - a real bus. Created by
> > us, or someone else.
> >
> > onMessage(msg) {
> > if(subscriptionMap.contains(msg.getTarget())) {
> > subscriptionMap.get(msg.getTarget()).forEach(sub -> sub.writeMsg()));
> > }
> > }
> >
> > addSubscriber(Subscriber sub, String target) { subscriptionMap.add(target,
> > sub); }
> >
> > Millions of msgs per second should be quite normal performance per node.
> > Add
> > thread notify & waits and have fun with dispatcher/multiplexer nirvana .
> > VertX/Guava/Hazelcast/etc have event busses also, but in the end this is
> > very simple process and should be part of the bus - not part of the
> > metrics.
> >
> > Components should not create the event bus, that should have been the whole
> > point of hawkular-bus. If it can't do that, then it needs to be replaced.
> >
> > - Micke
> >
> > ----- Original Message -----
> > From: "Jay Shaughnessy" <jshaughn at redhat.com> To: "Discussions around
> > Hawkular development" <hawkular-dev at lists.jboss.org> Sent: Tuesday, August
> > 9, 2016 12:00:24 AM
> > Subject: [Hawkular-dev] metrics on the bus
> >
> >
> > Lucas and I were talking over jira [1] which has to do with
> > metrics/alerting
> > scale. This was discussed a bit on IRC recently as well. Today, metrics
> > publishes all datapoints to the bus (metrics and avail go to different
> > topics). The only consumer of that data is alerting, and it consumes a
> > small
> > fraction of the total data (actually it consumes none of it OOB at the
> > moment, but that will hopefully change as Lucas's alerting work comes on
> > line in MIQ).
> >
> > Although in its purest form this publish-it-all is the essence of bus
> > publishing, we both feel it's an unnecessary waste of resources, as metrics
> > can reach very high volume. There are a few approaches to reducing the
> > publishing/filtering that we're currently doing. The options we discussed
> > boil down to:
> >
> >
> > * No Publishing
> >
> >
> > * Just query metrics for the data needed for alerting (or whatever
> > other external use we may have for the data)
> > * This is essentially a polling approach with frequent polling
> > * Demand Publishing
> >
> >
> > * The "just tell me what movie you want to see" approach
> > * Let clients request the metric ids it wants published to the bus
> >
> >
> > I'm purposefully not going into much detail at this point. I'd rather we
> > talk
> > out a preferred approach between these two, or something not presented. But
> > we'd like to move away from the current publish-it-all approach.
> > [1] https://issues.jboss.org/browse/HWKALERTS-118 .
> >
> > _______________________________________________
> > hawkular-dev mailing list hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > _______________________________________________
> > hawkular-dev mailing list hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> >
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> > _______________________________________________
> > hawkular-dev mailing list
> > hawkular-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hawkular-dev
> >
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>
More information about the hawkular-dev
mailing list