<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<div class="moz-cite-prefix">On 02/18/2015 04:17 PM, Randall Hauch
wrote:<br>
</div>
<blockquote
cite="mid:B3E0D082-5F7F-498D-A4AF-CA5399E5A987@redhat.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Feb 18, 2015, at 6:50 AM, Heiko Braun <<a
moz-do-not-send="true"
href="mailto:ike.braun@googlemail.com" class="">ike.braun@googlemail.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252" class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;" class=""><br
class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On 18 Feb 2015, at 13:43, John Sanda
<<a moz-do-not-send="true"
href="mailto:jsanda@redhat.com" class="">jsanda@redhat.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class=""><span style="font-family: Helvetica;
font-size: 16px; font-style: normal; font-variant:
normal; font-weight: normal; letter-spacing:
normal; line-height: normal; orphans: auto;
text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows:
auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px; float: none;
display: inline !important;" class="">I think that
Spark's streaming API, particularly the window
operations, could be an effective way to do
computations in real time as data as ingested</span></div>
</blockquote>
</div>
<br class="">
<div class="">+1</div>
<div class=""><br class="">
</div>
<div class="">not only for processing the streams, but
also for any kind of post processing needed. plus it
would supply the abstractions to run computations across
large number of nodes.</div>
<div class=""><br class="">
</div>
</div>
</div>
</blockquote>
<br class="">
</div>
<div>Exactly. Use Spark Streaming or even Storm would increase the
installation and operational complexity </div>
</blockquote>
We have a good experience of users being turned down by installation
and operational complexity, so no matter the "but" part (even though
that sounds interesting), we would need to find a proper solution to
remove it / reduce it.<br>
<br>
It needs to be easy to install in small environments and able to
scale when needed/wanted. Scaling by adding homogeneous nodes would
help.<br>
<br>
I have no experience with Spark/Storm, what is the burden on
installation and operational complexity ? <br>
<br>
Thomas<br>
<br>
<br>
<br>
<blockquote
cite="mid:B3E0D082-5F7F-498D-A4AF-CA5399E5A987@redhat.com"
type="cite">
<div>but it would give you a lot of really easy management of your
workflow. Each of the stream processors consume a stream and do
one thing (e.g., aggregate the last 60 seconds of raw data, or
aggregate the last 60 minutes of either the raw data or the
60-second windows, etc.). The different outputs can still get
written to whatever storage you want; stream-oriented processing
just changes *how* you process the incoming data.</div>
<div><br class="">
</div>
<div>An alternative approach is to use Apache Kafka directly.
There are pros and cons, but the benefit is that the services
that do the computations would be microservices (no, really -
just really small, single-threaded processes that do a single
thing) that could be easily deployed across the cluster. If
anyone is interested this approach, ping me and I’ll point you
to a prototype that does this (not for analytics).</div>
<div><br class="">
</div>
<div>BTW, a stream-processing approach does not limit you to live
data. In fact, quite the opposite. Many people use stream
processing for ingesting large volumes of live data, but lots of
other people use it in “big data” as an alternative to batch
processing (often map-reduce).</div>
<br class="">
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
hawkular-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:hawkular-dev@lists.jboss.org">hawkular-dev@lists.jboss.org</a>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/hawkular-dev">https://lists.jboss.org/mailman/listinfo/hawkular-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>