[Hawkular-dev] Clustering and Scalability

[John Sanda]

I am not sure why you think we are looking for YARN. I think we have been pretty consistent about needing to keep external dependencies to a minimum. Granted I have said that Spark is something we ought to look at, but I have also said that its biggest drawback is the additional external dependencies. As for Cassandra, we introduced that in RHQ after a lengthy, thorough, collaborative analysis because it was clear that the RDBMS was not the best fit for metrics. I think we should take a similar approach when considering new/other external dependencies. Trying to minimize external dependencies does not preclude the need for batch operations, streaming, resilience, scalability, etc. It may however make those features more difficult to implement.


> On Mar 10, 2015, at 8:07 AM, Michael Burman <miburman at redhat.com> wrote:
> 
> Hi,
> 
> So what we're looking for is basically YARN or equivalent. Batch 
> operations, durable streaming applications, data warehousing, 
> resilience, scalability.. All of the tools that provide us such features 
> are quite heavyweight solutions. If we keep adding small solutions such 
> as "Cassandra", "Spark" or others, we'll soon need to containerize them 
> to stick together and so on.. and eventually we'll have a Hadoop.
> 
> And I hope we won't reinvent such platform. So more likely we'll need to 
> scale down what we really really need.
> 
>   -  Micke
> 
> On 09.03.2015 17:27, John Sanda wrote:
>> For metrics it is more than a load balancer and scaling Cassandra. We have to also consider things like,
>> 
>> * computing aggregates
>> * changing data retention
>> * deleting metrics
>> 
>> We need to decide how these types of work should be distributed across hawkular metrics nodes.
>> 
>>> On Mar 9, 2015, at 10:53 AM, Thomas Segismont <tsegismo at redhat.com> wrote:
>>> 
>>> I can't answer for Hawkular as a whole. But for the metrics component,
>>> the principle is to grow the backend Cassandra and Wildfly servers
>>> clusters. You'll need a load balancer in front of the Wildfly servers.
>>> 
>>> There's no documentation yet. Cassandra cluster administration is
>>> extensively documented though.
>>> 
>>> Le 09/03/2015 14:50, Matt Wringe a écrit :
>>>> Hey,
>>>> 
>>>> I am trying to figure out how clustering and scalability works in
>>>> Hawkular/Hawkular metrics. Do we have any documentation on this?
>>>> 
>>>> Thanks,
>>>> 
>>>> Matt Wringe
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> hawkular-dev mailing list
>>>> hawkular-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>> 
>>> _______________________________________________
>>> hawkular-dev mailing list
>>> hawkular-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>> 
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> 
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev