[JBoss JIRA] (TEIID-5620) Provide flattened jmx metrics
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5620?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5620:
---------------------------------------
>From https://github.com/prometheus/jmx_exporter compositedata is fine, it just requires some additional keys in the export definition. I'll test that out as well. It likely means that we'll keep the existing admin/jmx bean structures and only add flattened values for the most important metrics.
> Provide flattened jmx metrics
> -----------------------------
>
> Key: TEIID-5620
> URL: https://issues.jboss.org/browse/TEIID-5620
> Project: Teiid
> Issue Type: Quality Risk
> Components: Server
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.1
>
>
> Providing JMX metrics via jolokia to Prometheus does not work well with complex types. We are providing several metric sets - engine stats, worker pool stats, requests, etc. as beans which needs to be flattened. Simple parameters are ok to the metrics methods, but simple results are required.
> We need to make sure that at least statistics are available on:
> * number of plans waiting to be processed - this will be a primary metric for determining horizontal scale out
> * amount /%of buffer memory used - this will be a primary metric for determining if pods need to be allocated with more memory/disk
> Ideally alerting could be available on percent utilization - but we could also add a metric such as out of disk count to provide a hard count of exceptional conditions.
> It's assumed that cpu utilization will be monitored from the pod itself - which will be a primary metric for determining if pods should be allocated with more cpu resources.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
6 years, 11 months
[JBoss JIRA] (TEIID-5631) Issues with ThreadReuseExecutor
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5631?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-5631.
-----------------------------------
Resolution: Done
Addressed the issues listed above. Sampling now should be thread safe / non-blocking.
> Issues with ThreadReuseExecutor
> -------------------------------
>
> Key: TEIID-5631
> URL: https://issues.jboss.org/browse/TEIID-5631
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.1
>
>
> In looking at flattening the metrics there are a couple of issues with the ThreadReuseExecutor:
> 1. The getStats method uses unsynchronized access to get the queue size.
> 2. The queue ordering prefers older work items to newer - even after repeated queuing.
> 3. Thread tracking could be more concurrent.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
6 years, 11 months
[JBoss JIRA] (TEIID-5631) Issues with ThreadReuseExecutor
by Steven Hawkins (Jira)
Steven Hawkins created TEIID-5631:
-------------------------------------
Summary: Issues with ThreadReuseExecutor
Key: TEIID-5631
URL: https://issues.jboss.org/browse/TEIID-5631
Project: Teiid
Issue Type: Bug
Components: Query Engine
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 12.1
In looking at flattening the metrics there are a couple of issues with the ThreadReuseExecutor:
1. The getStats method uses unsynchronized access to get the queue size.
2. The queue ordering prefers older work items to newer - even after repeated queuing.
3. Thread tracking could be more concurrent.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
6 years, 11 months
[JBoss JIRA] (TEIID-5613) upgrade to thorntail 2.3
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5613?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5613:
---------------------------------------
The code changes are in for master with some minor cleanup as well. I'll see where appropriate doc updates should be made and add a forward looking cautionary statement about thorntail 4.x.
> upgrade to thorntail 2.3
> ------------------------
>
> Key: TEIID-5613
> URL: https://issues.jboss.org/browse/TEIID-5613
> Project: Teiid
> Issue Type: Task
> Components: thorntail
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.1, tt-2.0.0
>
>
> Thorntail with THORN-2126 has intermediate versions, such as the 1.5.3 config-api, on wildfly 14 - but moved to WildFly 15 for the release. I'll experiment a little if Teiid 12 is compatible enough to build Thorntail 2.3 support directly. More than likely though we'll need to wait until Teiid 13 to sync back up.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
6 years, 11 months