[Metrics] How to react on low disk?
by Heiko W.Rupp
Hey,
<captain_obvious>
so for Hawkular-metrics (and Hawkular) we store the data in a Cassandra
database that puts files on a disk,
which can get full earlier than expected (and usually on week-ends). And
when the disk is full, Metrics does not like it.
</captain_obvious>
What can we do in this case?
I could imagine that on the C* nodes we run a script that uses df to
figure out the available space and tries
to run some compaction if space gets tight.
Of course that does not solve the issue per se, but should give some air
to breathe.
Right now I fear we are not able to reduce the ttl of a metric/tenant on
the fly and have metrics do the right thing - at least if I understood
yak correctly.
That script should possibly also send an email to an admin.
In case that we run Hawkular-full, we can determine the disk space and
feed that into Hawkular for Alerts to pick it up and then have the
machinery trigger the compaction and send the email.
6 years, 11 months
using hawkular wildfly agent as a custom java agent
by John Mazzitelli
This is for Matt, but figured post here for public consumption.
The question was asked yesterday, "Can we use the Hawkular WildFly Agent to monitor other things other than WildFly?"
I gave one answer, but forgot there is a second alternative.
====
The first answer that I gave is that you can use the Hawkular Wildfly Agent to collect JMX data via Jolokia interface. There is an integration in the agent that lets you define your resource and metric metadata and your JMX/Jolokia servers. As an example, see here: https://github.com/hawkular/hawkular-agent/blob/b52529823ca3c54d0b8b4aa56...
You define where your JMX servers are via the <remote-jmx> managed server like this:
<remote-jmx name="Remote JMX" enabled="false" resourceTypeSets="MainJMX,MemoryPoolJMX" url="http://localhost:8080/jolokia-war"/>
OK, that's the JMX integration. Maybe useful, maybe not. But I mention it just in case.
====
The second alternative I forgot to mention was the ability for any component running in WildFly to obtain a Hawkular Agent proxy via JNDI and use that proxy that store inventory and metrics into the Hawkular Server.
There is an example WAR module in the agent git repo that demonstrates how to obtain the proxy via JNDI and how to store inventory and metrics - see here: https://github.com/hawkular/hawkular-agent/tree/master/hawkular-wildfly-a...
This is just a simple WAR with a servlet. But it shows how a component can get the agent proxy via JNDI here:
https://github.com/hawkular/hawkular-agent/blob/master/hawkular-wildfly-a...
Here's code that shows the servlet doing things like sending metrics, avail, and creating resources:
https://github.com/hawkular/hawkular-agent/blob/master/hawkular-wildfly-a...
No one is using this yet. So there may be issues I am not aware of, but we have integration tests that show this working.
This was put together with the anticipation of someone asking for this capability - that is, "can the agent be used to collect metrics for other things other than WildFly". Essentially, this just gives you a skeleton Java agent that you can extend to collect your own metrics and inventory. So you can write a WAR or EAR, deploy it in any WildFly that has an agent subsystem, and your EAR/WAR can be used as an "agent" for your custom stuff.
Again, maybe useful, maybe not. But I mention it just in case.
6 years, 11 months
ruby-to-py
by John Mazzitelli
The ruby client is getting a lot of work and seems to be coming into shape.
What are the chances of us being able to take that ruby client and "copy" it to make a python client so we can have both ruby and py support?
6 years, 12 months
Test coverage
by Heiko W.Rupp
Hey,
I think we should start looking at and recording test coverage
of our various Hawkular artifacts and also "enforce" them like
failing pull-requests when coverage is going below a certain
threshold.
We should have done that long time ago, but better late then
never ;-)
Heiko
7 years
Fwd: [aerogear-dev] Artemis bug on WildFly-10
by Heiko W.Rupp
Also as FYI in case we see something similar.
> From: Matthias Wessendorf <matzew(a)apache.org>
> To: AeroGear Developer Mailing List <aerogear-dev(a)lists.jboss.org>
> Subject: [aerogear-dev] Artemis bug on WildFly-10
> Date: Thu, 24 Mar 2016 00:42:46 +0100
>
> Hi,
>
> more as a FYI....
>
> I am in the process of updating our Docker -DEV image to WF/10 (see
> [1]),
> and I noticed a bug in Artemis:
>
>
> Caused by: javax.jms.JMSException: Failed to create session factory
> at
> org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:727)
> at
> org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:233)
> at
> org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:229)
> at
> org.jboss.aerogear.unifiedpush.message.util.JmsClient$JmsReceiver.from(JmsClient.java:173)
> ... 172 more
> Caused by:
> ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT
> message=AMQ119013: Timed out waiting to receive cluster topology.
> Group:null]
> at
> org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:813)
> at
> org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:724)
> ... 175 more
> Now, this issue is addressed by
> https://issues.apache.org/jira/browse/ARTEMIS-385 and fixed in
> ActiveMQ-Artemis 1.3.0 (unreleased), but WildFly 10.x is (currently)
> on
> their 1.1.0 version.
>
> Worth to mention; I think I am unable to reproduce this directly on my
> Mac,
> just w/ CentOS7 and WF-10 (via ), using this image:
> https://github.com/jboss-dockerfiles/wildfly/tree/10.0.0.Final
>
>
> Cheers,
> Matthias
>
> [1] https://github.com/matzew/dockerfiles/tree/WF_10_ups_120
>
>
> --
> Matthias Wessendorf
>
> blog: http://matthiaswessendorf.wordpress.com/
> sessions: http://www.slideshare.net/mwessendorf
> twitter: http://twitter.com/mwessendorf
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
7 years
Cancelled: Discussion, Planning, and status on Testing Openshift & Hawk-Metrics Integration for Openshift 3.1
by Michael Foley
You have been removed from the attendee list by the organizer.
-----Original Invite-----
The following meeting has been modified:
Subject: Discussion, Planning, and status on Testing Openshift & Hawk-Metrics Integration for Openshift 3.1
Organizer: "Michael Foley" <mfoley(a)redhat.com>
Location: Bluejeans http://www.bluejeans.com/mfoley51
Time: 3:00:00 PM - 3:30:00 PM GMT -05:00 US/Canada Eastern [MODIFIED]
Recurrence : Every 1 week(s) on No end date Effective Sep 9, 2015
Required: pruan(a)redhat.com; mmahoney(a)redhat.com; vnguyen(a)redhat.com; snegrea(a)redhat.com; jsanda(a)redhat.com; mwringe(a)redhat.com; jkandasa(a)redhat.com
Optional: jon-qa-list(a)redhat.com; jboss-on-team(a)redhat.com; hawkular-dev(a)lists.jboss.org
~ ~ ~ ~ ~ ~ ~ ~ ~
Hi,
Let's have a discussion and planning session on testing Openshift & Hawkular Integration!
Let's use this etherpad to coordinate the discussion -->> http://pad.engineering.redhat.com/Management-nextAndOpenshiftTestPlanning
5 Point Plan for Openshift 3.1 GA
* Unit tests .... owned by Hawk-Metrics developers
* Integration tests ... owned by Hawk-Metrics developers and Hawk-Metrics QE
* Performance CI on Hawk-Metrics (this one is actually new and was not discussed on Wednesday , but I now see it makes sense)
* Functional Integration tests on Hawk-Metrics latest + Openshift Origin latest
*
Funtional/UI .... Cucumber/Ruby tests ...owned by Openshift QE * Functional/Rest ... Cucumber/Ruby tests ... owned by Openshift QE
* Performance & Scalability .... owned by Openshift QE
Regards,
Michael Foley
QE Supervisor, Middleware BU
7 years
