[Hawkular-dev] Could not connect to Cassandra ... - does it have to be a WARN with a stack trace?

[Jay Shaughnessy]

On 7/27/2015 9:53 AM, Matt Wringe wrote:
> There are a couple of things here.
>
> One is the embedded Cassandra. I don't know if its a dependency issue in
> that we are not defining the embedded Cassandra as a dependency properly
> or if its something along the lines of that the dependency is started,
> but not yet able to accept connections yet. Either way, it looks like we
> need to figure that out and handle the warning in the logs better (which
> could be fixing the dependency management, or making it info level for
> the first couple of failures).

I don't think It's feasible to add a dependency on embedded cassandra 
because we want to support embedded or remote and jboss-all.xml does not 
support optional deployment dependencies (afaik).  So, I think the best 
you can do with the nest deployment is what hawkular-alerts-metrics.war 
does, it depends on hawkular-metrics-api-jaxrs.war.  But it may be the 
case that you don't want your deployment to know it's part of hawkular 
at all. In that case adding a jboss-all.xml means you need to build 
specifically for hawkular.

If we want to suppress the messages the for some fairly large period 
either don't log or log info/debug.

>
> The other issue is when we are using an external Cassandra and its not
> yet available to take connections when Hawkular is started. In Hawkular
> Metrics anyways, it will start and throw the warning messages in the
> logs until Cassandra is fully running. This is done so that we can start
> Hawkular Metrics and Cassandra at the same time and not have to worry
> about the order of things. This is really handy if Cassandra and
> Hawkular Metrics are configured as services to start when the machine is
> rebooted, and also when dealing with containers (especially since
> containers don't have the same concept of dependencies).
>
> But maybe these should be changed to 'info' level for the first while
> and only then transformed into 'warnings'. Or handled in some other
> manner. Its a bit weird since when starting containers at the same time,
> the initial failures are expected, but if they persist, it usually means
> the Cassandra container was not started or failed to startup properly.

+1, same sort of thing.

>
> On 27/07/15 05:55 AM, Lucas Ponce wrote:
>> +1
>>
>> The WARNING message is correct as the component cannot connect to Cassandra, so, I don't remove those messages.
>>
>> But as TH says, it's a matter of how to define the embedded cassandra in this scenario to be up and running before that the components.
>>
>> Lucas
>>
>> ----- Original Message -----
>>> From: "Thomas Heute" <theute at redhat.com>
>>> To: "Discussions around Hawkular development" <hawkular-dev at lists.jboss.org>
>>> Sent: Monday, July 27, 2015 11:42:04 AM
>>> Subject: Re: [Hawkular-dev] Could not connect to Cassandra ... - does it have to be a WARN with a stack trace?
>>>
>>> Glad I am not the only one that finds those messages annoying :)
>>>
>>> Indeed, this is an Hawkular issue with embedded Cassandra.
>>> We should fix it by properly defining the startup dependencies, we may also
>>> think of keeping "embedded Cassandra" as a separate process.
>>>
>>> The idea of "embedding Cassandra" is that we don't need to ask users to go
>>> download it, "install it" and run it.
>>> But we could consider having a Cassandra in a different process (configured
>>> for our needs such as the data location, port and other settings) that is
>>> required to be started prior to run Hawkular (Hawkular would need to fail
>>> quickly and clearly if the user "forgot" to start Cassandra).
>>>
>>> PS: I am not advocating for one or the other solution as I don't know the
>>> advantages/drawbacks of both solutions
>>>
>>> Thomas
>>>
>>>
>>>
>>>
>>>
>>> On 07/27/2015 11:11 AM, Michael Burman wrote:
>>>> Hi,
>>>>
>>>> And how would you distinguish between real warning and incorrect one? This
>>>> message is not printed when Hawkular-Metrics starts standalone andshould
>>>> not be removed. This is a container issue in Hawkular and should be fixed
>>>> there (so that no such error happens) instead of breaking the logging in
>>>> metrics. Most likely this happens with embedded-cassandra I guess?
>>>>
>>>>      - Micke
>>>>
>>>> ----- Original Message -----
>>>> From: "Peter Palaga" <ppalaga at redhat.com>
>>>> To: "Discussions around Hawkular development"
>>>> <hawkular-dev at lists.jboss.org>
>>>> Sent: Monday, July 27, 2015 11:59:27 AM
>>>> Subject: [Hawkular-dev] Could not connect to Cassandra ... - does it have
>>>> to be a WARN with a stack trace?
>>>>
>>>> Hi *,
>>>>
>>>> There are several occurrences of this in every Hawkular start log:
>>>>
>>>> WARN  [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle]
>>>> (metricsservice-lifecycle-thread) Could not connect to Cassandra cluster
>>>> - assuming its not up yet:
>>>> com.datastax.driver.core.exceptions.NoHostAvailableException: All
>>>> host(s) tried for query failed (tried: /127.0.0.1:9042
>>>> (com.datastax.driver.core.TransportException: [/127.0.0.1:9042] Cannot
>>>> connect))
>>>>
>>>> plus the stack trace.
>>>>
>>>> So given that this happens during every HK startup, could we not
>>>> classify it as normal and change it to INFO without the stack trace?
>>>>
>>>> I am ready to prepare a PR unless somebody raises a hand against that.
>>>>
>>>> Thanks,
>>>>
>>>> Peter
>>>> _______________________________________________
>>>> hawkular-dev mailing list
>>>> hawkular-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>> _______________________________________________
>>>> hawkular-dev mailing list
>>>> hawkular-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>>
>>> _______________________________________________
>>> hawkular-dev mailing list
>>> hawkular-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>>>
>> _______________________________________________
>> hawkular-dev mailing list
>> hawkular-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hawkular-dev
> _______________________________________________
> hawkular-dev mailing list
> hawkular-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hawkular-dev
>