[aerogear-dev] Simplify the metrics for sanity

[Matthias Wessendorf]

Hi,

we do have a problem w/ our current metrics processing. It's complicated
(lot's of CDI events and two different JMS messaging approaches...) and
also slow (JPQL/JDBC) and it does consume a lot of memory and processing
time. This is leading to bugs (incorrect stats) and eventually causes down
times, due to heavy processing.

I'd like to dramatically simplify our metrics processing... to something
like:
Success -> could connect to 3rd party, to deliver tokens
Failure -> something went wrong when talking to 3rd party service.


Right now we do have metrics on push delivery:
Pending -> the submission to the 3rd party provider is in flight
Success -> we were able to connect, and could deliver *something*
Failure -> something obvious, like invalid certificate (APNs), no
connection to 3rd party possible, etc

Besides that, we also do a count on targeted devices. I think there is not
really a huge value. For instance if APNs rejects some tokens, we do not
track those, we just show how many tokens our DB did find, not more. We
don't show any of real interest. We could improve this (see below), but I
doubt that the current implementation is able to handle this well.

Also, on Android/FCM the numbers are even worse. We do, internally,
leverage their topics, so we usually end up sending exactly one push to
FCM, regardless of how many Android device-tokens we have in the DB. The
counter says 1 (one), because the server did target one topic (not n
devices).

So, for now, I'd like to dramatically simplify the code, and go with the
above Success/Failure solution.

However, I honestly think in the long run, we should get something
pluggable, that allows us to process the metrics independently, outside of
the UPS code base. I think my previous Kafka mail is addressing this
partially: The actual response and details about the push job should be
logged to some Kafka system, and an independent process should be able to
process those.

This will give us much more freedom and flexibility. Perhaps also, in the
future, we want some different stats, and something like Prometheus
/Grafana:
https://prometheus.io/docs/visualization/grafana/

A more flexible system, with independent metrics 'calculation' processing
will help us here.

Any thoughts?

-Matthias



-- 
Matthias Wessendorf

blog: http://matthiaswessendorf.wordpress.com/
twitter: http://twitter.com/mwessendorfa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/aerogear-dev/attachments/20170524/22d4537d/attachment.html