<div dir="ltr"><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">Hi,</p><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">with the new APNs HTTP/2 APIs, and our usage of Pushy, we are able to get a way more finegrain knowledge if Apple did accept (for further processing) or reject a messages, on a per device_token level!</p><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">For instance, if we have a push with 5000 targeted devices, we are now able to say that 5 tokens, for instances failed, but APNs was happy to accept push request for the other 4995 devices (Note: this does NOT mean they actually arrive at the device, just that apple accepted them for further processing).</p><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">Now, this, for APNs, gives us much more flexiblity handling our metrics!</p><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">In our code, here, we do read <em style="box-sizing:border-box">each</em> token request from APNs in here: <a href="https://github.com/aerogear/aerogear-unifiedpush-server/blob/20831d96196663349c96da6b5fe11aef65cacf59/push/sender/src/main/java/org/jboss/aerogear/unifiedpush/message/sender/apns/PushyApnsSender.java#L130-L147" style="box-sizing:border-box;background-color:transparent;color:rgb(3,102,214)">https://github.com/aerogear/aerogear-unifiedpush-server/blob/20831d96196663349c96da6b5fe11aef65cacf59/push/sender/src/main/java/org/jboss/aerogear/unifiedpush/message/sender/apns/PushyApnsSender.java#L130-L147</a></p><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">So here, we could simply send the result, on a per token base, to a (Kafka) topic, like:</p><pre style="box-sizing:border-box;font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:13.6px;margin-top:0px;margin-bottom:16px;font-stretch:normal;line-height:1.45;word-wrap:normal;padding:16px;overflow:auto;background-color:rgb(246,248,250);border-radius:3px;color:rgb(36,41,46)"><code style="box-sizing:border-box;font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:13.6px;padding:0px;margin:0px;background:transparent;border-radius:3px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit;word-wrap:normal">...
if (pushNotificationResponse.isAccepted()) {
logger.trace("Push notification for '{}' (payload={})", deviceToken, pushNotificationResponse.getPushNotification().getPayload());
producer.send(jobID, "Success"); // sends to "push_messages" topic
} else {
final String rejectReason = pushNotificationResponse.getRejectionReason();
logger.trace("Push Message has been rejected with reason: {}", rejectReason);
producer.send(jobID, "Rejected"); // sends "push_messages" topic
...
}
</code></pre><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">Now, this sends all to one topic, and we could be using, somewhere, Kafka Stream API, to perform some processing of the source, and calculate some stats on that, like:</p><pre style="box-sizing:border-box;font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:13.6px;margin-top:0px;margin-bottom:16px;font-stretch:normal;line-height:1.45;word-wrap:normal;padding:16px;overflow:auto;background-color:rgb(246,248,250);border-radius:3px;color:rgb(36,41,46)"><code style="box-sizing:border-box;font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:13.6px;padding:0px;margin:0px;background:transparent;border-radius:3px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit;word-wrap:normal">KStreamBuilder builder = new KStreamBuilder();
// read from the topic that contains all messages, for all jobs
final KStream<String, String> source = builder.stream("push_messages");
// some simple processing, and grouping by key, applying a predicate and send to three "analytic" topic:
final KTable<String, Long> successCountsPerJob = source.filter((key, value) -> value.equals("Success"))
.groupByKey()
.count("successMessagesPerJob");
successCountsPerJob.to(Serdes.String(), Serdes.Long(), "successMessagesPerJob");
final KTable<String, Long> failCountsPerJob = source.filter((key, value) -> value.equals("Rejected"))
.groupByKey()
.count("failedMessagesPerJob");
failCountsPerJob.to(Serdes.String(), Serdes.Long(), "failedMessagesPerJob");
source.groupByKey()
count("totalMessagesPerJob")
.to(Serdes.String(), Serdes.Long(), "totalMessagesPerJob");
</code></pre><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">The above performs some functional processing of the single source of truth, based on different assumptions.</p><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">If one would have a simple consumer on each of these three "analytic" topics, a simple logging output would be:</p><pre style="box-sizing:border-box;font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:13.6px;margin-top:0px;margin-bottom:16px;font-stretch:normal;line-height:1.45;word-wrap:normal;padding:16px;overflow:auto;background-color:rgb(246,248,250);border-radius:3px;color:rgb(36,41,46)"><code style="box-sizing:border-box;font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:13.6px;padding:0px;margin:0px;background:transparent;border-radius:3px;word-break:normal;border:0px;display:inline;overflow:visible;line-height:inherit;word-wrap:normal">2017-05-16 13:42:48,763 INFO successMessagesPerJob: 2 - jobID: XXX
2017-05-16 13:42:48,764 INFO totalMessagesPerJob: 3 - jobID: XXX
2017-05-16 13:42:48,764 INFO failedMessagesPerJob: 1 - jobID: XXX
</code></pre><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">since for the GSoC we do have two students, working on Kafka and HBase improvements for UPS, I wanted to share this quick prototype, as food for thoughts.</p><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">Of course, each of these 'filtered' consumers could than eventually store the result somewhere else.</p><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">With this approach, Kafka would be come the hub (or data pipeline) for our metrics, with stream processing and different consumers to deal with the results of interest</p><p style="box-sizing:border-box;margin-top:0px;margin-bottom:16px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px">Any comments or other thoughts?</p><p style="box-sizing:border-box;margin-top:0px;color:rgb(36,41,46);font-family:-apple-system,blinkmacsystemfont,"segoe ui",helvetica,arial,sans-serif,"apple color emoji","segoe ui emoji","segoe ui symbol";font-size:16px;margin-bottom:0px">-Matthias</p><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div>Matthias Wessendorf <br><br>blog: <a href="http://matthiaswessendorf.wordpress.com/" target="_blank">http://matthiaswessendorf.wordpress.com/</a><br>twitter: <a href="http://twitter.com/mwessendorf" target="_blank">http://twitter.com/mwessendorf</a></div></div></div>
</div>