[Hawkular-dev] metrics on the bus

Tue Aug 9 13:39:02 EDT 2016

just for the record, we can't remove hawkular-bus - we use it for other things than just metrics :)

----- Original Message -----
> Hi,
> 
> Every inserted metric has to be processed inside metrics anyway, so it's an
> event for us. And we can't tell if some system wants it. The filtering has
> to be done somewhere and it's certainly not the metrics' job and event bus
> shouldn't work in that way either. Event bus should be done the one doing
> the filtering and routing, not the components producing the data. Why would
> we do the things backwards? We should remove the whole metrics-bus component
> if we do something on the metrics side - it has no purpose anymore in any
> sane architectural view.
> 
> There's no reason to push this work to metrics without removing hawkular-bus.
> It makes more sense to integrate alerts to metrics directly then - and I
> thought we wanted separated components.
> 
> Is there a reason we can't use known-to-work-architecture?
> 
> https://github.com/google/guava/wiki/EventBusExplained
> 
> If we add the functionality to metrics, it should happen with this (as we
> already have Guava and vertx isn't an option). And then it should remove
> hawkular-bus as it's been replaced.
> 
>   - Micke
> 
> ----- Original Message -----
> From: "Jay Shaughnessy" <jshaughn at redhat.com>
> To: hawkular-dev at lists.jboss.org
> Sent: Tuesday, August 9, 2016 4:12:24 PM
> Subject: Re: [Hawkular-dev] metrics on the bus
> 
> 
> An event bus is for publishing events and I'm not sure every datum is an
> event of any significance. The storage of an interesting datum could be an
> event, and what is interesting can't be determined by the storage mechanism
> unless told. It's not unlike adding a trigger to a db, or registering a
> listener to be informed of significant happenings among a larger set of
> happenings. So, to a degree I agree with Micke, publishing all events to the
> bus is good. But that leaves the semantic of choosing what an event is, and
> I'd say every datum is not an event. And I'd also say that just because a
> bus can handle millions of events, there is a practicality to not doing a
> bunch of unnecessary work, and instead leaving that bandwidth available to
> be used for solving actual business problems.
> 
> I agree with Heiko that there are drawbacks to the polling approach (as
> usual). But it has the advantage of not needing any further work on the
> metrics side. But since Metrics already has hooks for bus publishing in
> hawkular, perhaps it wouldn't be hard to add some selection as well. I'm
> also fine with some sort of listener approach that leverages the co-location
> of components. Maybe some sort of Observer. Perhaps Micke or other metrics
> guys could comment on that possibility.
> 
> 
> On 8/9/2016 6:18 AM, Michael Burman wrote:
> 
> 
> 
> Hi,
> 
> Like I said in the IRC, I don't like either approach and both sort of piss on
> the well known EIP pattern. Since this problem has been solved ages ago, why
> are we going to reinvent it? Use a bus, and I mean - a real bus. Created by
> us, or someone else.
> 
> onMessage(msg) {
>  if(subscriptionMap.contains(msg.getTarget())) {
>    subscriptionMap.get(msg.getTarget()).forEach(sub -> sub.writeMsg()));
>  }
> }
> 
> addSubscriber(Subscriber sub, String target) { subscriptionMap.add(target,
> sub); }
> 
> Millions of msgs per second should be quite normal performance per node. Add
> thread notify & waits and have fun with dispatcher/multiplexer nirvana .
> VertX/Guava/Hazelcast/etc have event busses also, but in the end this is
> very simple process and should be part of the bus - not part of the metrics.
> 
> Components should not create the event bus, that should have been the whole
> point of hawkular-bus. If it can't do that, then it needs to be replaced.
> 
>   - Micke
> 
> ----- Original Message -----
> From: "Jay Shaughnessy" <jshaughn at redhat.com> To: "Discussions around
> Hawkular development" <hawkular-dev at lists.jboss.org> Sent: Tuesday, August
> 9, 2016 12:00:24 AM
> Subject: [Hawkular-dev] metrics on the bus
> 
> 
> Lucas and I were talking over jira [1] which has to do with metrics/alerting
> scale. This was discussed a bit on IRC recently as well. Today, metrics
> publishes all datapoints to the bus (metrics and avail go to different
> topics). The only consumer of that data is alerting, and it consumes a small
> fraction of the total data (actually it consumes none of it OOB at the
> moment, but that will hopefully change as Lucas's alerting work comes on
> line in MIQ).
> 
> Although in its purest form this publish-it-all is the essence of bus
> publishing, we both feel it's an unnecessary waste of resources, as metrics
> can reach very high volume. There are a few approaches to reducing the
> publishing/filtering that we're currently doing. The options we discussed
> boil down to:
> 
> 
>     * No Publishing
> 
> 
>         * Just query metrics for the data needed for alerting (or whatever
>         other external use we may have for the data)
>         * This is essentially a polling approach with frequent polling
>     * Demand Publishing
> 
> 
>         * The "just tell me what movie you want to see" approach
>         * Let clients request the metric ids it wants published to the bus
> 
> 
> I'm purposefully not going into much detail at this point. I'd rather we talk
> out a preferred approach between these two, or something not presented. But
> we'd like to move away from the current publish-it-all approach.
> [1] https://issues.jboss.org/browse/HWKALERTS-118 .
> 
> _______________________________________________
> hawkular-dev mailing list hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> _______________________________________________
> hawkular-dev mailing list hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>