[JBoss JIRA] (TEIIDSB-23) Document image generation options
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIIDSB-23?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIIDSB-23:
---------------------------------------
> we should target grafana for this example, leave dashboard for a later discussion with others. What do we need to do for this?
Something like: https://github.com/openshift/origin/tree/master/examples/grafana
It sounds like teams will be responsible for defining their own dashboards.
> Document image generation options
> ---------------------------------
>
> Key: TEIIDSB-23
> URL: https://issues.jboss.org/browse/TEIIDSB-23
> Project: Teiid Spring Boot
> Issue Type: Task
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: Q119
>
>
> We need to document / validate all relevant image options:
> - inclusion of agent bond or other mechanism for jmx exposure to prometheus. There may also be related service annotations
> - annotations for 3scale for rest and openapi exposure of odata
> - any common config options - disk buffer memory, max active plans / engine threads / connection pool sizes. Ideally the buffer manager heap should auto-configure and we should probably always use off-heap for the fixed memory buffer.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 10 months
[JBoss JIRA] (TEIID-5658) PG transport can deliver messages out of order
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5658?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5658:
---------------------------------------
No it is quite strange making assumptions based upon our logs or looking at the code. It's not intuitive that netty channel writes have ordering issues.
> PG transport can deliver messages out of order
> ----------------------------------------------
>
> Key: TEIID-5658
> URL: https://issues.jboss.org/browse/TEIID-5658
> Project: Teiid
> Issue Type: Bug
> Components: ODBC
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.1, 12.0.1, 11.2.3
>
>
> Although it is not apparent in our logs, we are delivering response messages out of order.
> From our logs we have:
> 2019-02-14 08:18:12.476 org.teiid.ODBC TRACE NIO1 invoking server method: prepare [, select 3 from pg_class where relname='pg_class', []]
> 2019-02-14 08:18:12.476 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f prepareCompleted []
> 2019-02-14 08:18:12.48 org.teiid.ODBC TRACE NIO1 invoking server method: bindParameters [, , [], 0, null, UTF-8]
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f bindComplete null
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking server method: getResultSetMetaDataDescription []
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f sendResultSetDescription [[org.teiid.odbc.PGUtil$PgColInfo@333d1d1b], null]
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking server method: execute [, 0]
> 2019-02-14 08:18:12.484 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f sendResults [null, org.teiid.jdbc.ResultSetImpl@4516a1a2, [org.teiid.odbc.PGUtil$PgColInfo@333d1d1b], org.teiid.client.util.ResultsFuture@1e98b9de, FORWARD, -1, false, null]
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE Worker2_QueryProcessorQueue211 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f sendCommandComplete [select 3 from pg_class where relname='pg_class', [1]]
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE NIO1 invoking server method: sync null
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f ready [false, false]
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE Worker0_QueryProcessorQueue212 invoking client method: org.teiid.transport.ODBCClientInstance@3e802be1 ready [false, false]
> At the network level we see ready delivered before the data row.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 10 months
[JBoss JIRA] (TEIID-5658) PG transport can deliver messages out of order
by Ramesh Reddy (Jira)
[ https://issues.jboss.org/browse/TEIID-5658?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-5658:
-------------------------------------
(y) I am sure this was not easy.
> PG transport can deliver messages out of order
> ----------------------------------------------
>
> Key: TEIID-5658
> URL: https://issues.jboss.org/browse/TEIID-5658
> Project: Teiid
> Issue Type: Bug
> Components: ODBC
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.1, 12.0.1, 11.2.3
>
>
> Although it is not apparent in our logs, we are delivering response messages out of order.
> From our logs we have:
> 2019-02-14 08:18:12.476 org.teiid.ODBC TRACE NIO1 invoking server method: prepare [, select 3 from pg_class where relname='pg_class', []]
> 2019-02-14 08:18:12.476 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f prepareCompleted []
> 2019-02-14 08:18:12.48 org.teiid.ODBC TRACE NIO1 invoking server method: bindParameters [, , [], 0, null, UTF-8]
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f bindComplete null
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking server method: getResultSetMetaDataDescription []
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f sendResultSetDescription [[org.teiid.odbc.PGUtil$PgColInfo@333d1d1b], null]
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking server method: execute [, 0]
> 2019-02-14 08:18:12.484 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f sendResults [null, org.teiid.jdbc.ResultSetImpl@4516a1a2, [org.teiid.odbc.PGUtil$PgColInfo@333d1d1b], org.teiid.client.util.ResultsFuture@1e98b9de, FORWARD, -1, false, null]
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE Worker2_QueryProcessorQueue211 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f sendCommandComplete [select 3 from pg_class where relname='pg_class', [1]]
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE NIO1 invoking server method: sync null
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f ready [false, false]
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE Worker0_QueryProcessorQueue212 invoking client method: org.teiid.transport.ODBCClientInstance@3e802be1 ready [false, false]
> At the network level we see ready delivered before the data row.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 10 months
[JBoss JIRA] (TEIIDSB-23) Document image generation options
by Ramesh Reddy (Jira)
[ https://issues.jboss.org/browse/TEIIDSB-23?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIIDSB-23:
-------------------------------------
> The 3 scale team is promoting a Prometheus instance per openshift cluster
In this example is IMO we should not be concerned about a new instance of Prometheus, this is mentioned in README as an example to showcase, it is expected that user in a different environment uses probably an existing instance. We need to just show what does it take to expose metrics.
> We need to validate that our pods at host:8778/metrics
I changed to port to 9779 and verified host:9779/metrics to match the container.
> Based upon that we can change the
I have no idea, go ahead and change it accordingly
> I'm not sure yet about the expectation for grafana instances
we should target grafana for this example, leave dashboard for a later discussion with others. What do we need to do for this?
> Document image generation options
> ---------------------------------
>
> Key: TEIIDSB-23
> URL: https://issues.jboss.org/browse/TEIIDSB-23
> Project: Teiid Spring Boot
> Issue Type: Task
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: Q119
>
>
> We need to document / validate all relevant image options:
> - inclusion of agent bond or other mechanism for jmx exposure to prometheus. There may also be related service annotations
> - annotations for 3scale for rest and openapi exposure of odata
> - any common config options - disk buffer memory, max active plans / engine threads / connection pool sizes. Ideally the buffer manager heap should auto-configure and we should probably always use off-heap for the fixed memory buffer.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 10 months
[JBoss JIRA] (TEIID-5658) PG transport can deliver messages out of order
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIID-5658?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-5658.
-----------------------------------
Resolution: Done
Addressed by using the event loop executor - which does guarantee order. Also added synchronization in several places to help enforce usage assumptions.
> PG transport can deliver messages out of order
> ----------------------------------------------
>
> Key: TEIID-5658
> URL: https://issues.jboss.org/browse/TEIID-5658
> Project: Teiid
> Issue Type: Bug
> Components: ODBC
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 12.1, 12.0.1, 11.2.3
>
>
> Although it is not apparent in our logs, we are delivering response messages out of order.
> From our logs we have:
> 2019-02-14 08:18:12.476 org.teiid.ODBC TRACE NIO1 invoking server method: prepare [, select 3 from pg_class where relname='pg_class', []]
> 2019-02-14 08:18:12.476 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f prepareCompleted []
> 2019-02-14 08:18:12.48 org.teiid.ODBC TRACE NIO1 invoking server method: bindParameters [, , [], 0, null, UTF-8]
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f bindComplete null
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking server method: getResultSetMetaDataDescription []
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f sendResultSetDescription [[org.teiid.odbc.PGUtil$PgColInfo@333d1d1b], null]
> 2019-02-14 08:18:12.481 org.teiid.ODBC TRACE NIO1 invoking server method: execute [, 0]
> 2019-02-14 08:18:12.484 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f sendResults [null, org.teiid.jdbc.ResultSetImpl@4516a1a2, [org.teiid.odbc.PGUtil$PgColInfo@333d1d1b], org.teiid.client.util.ResultsFuture@1e98b9de, FORWARD, -1, false, null]
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE Worker2_QueryProcessorQueue211 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f sendCommandComplete [select 3 from pg_class where relname='pg_class', [1]]
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE NIO1 invoking server method: sync null
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE NIO1 invoking client method: org.teiid.transport.ODBCClientInstance@6064753f ready [false, false]
> 2019-02-14 08:18:12.486 org.teiid.ODBC TRACE Worker0_QueryProcessorQueue212 invoking client method: org.teiid.transport.ODBCClientInstance@3e802be1 ready [false, false]
> At the network level we see ready delivered before the data row.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 10 months
[JBoss JIRA] (TEIIDSB-23) Document image generation options
by Steven Hawkins (Jira)
[ https://issues.jboss.org/browse/TEIIDSB-23?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIIDSB-23:
---------------------------------------
> I also added how to create the instance
The 3 scale team is promoting a prometheus instance per openshift cluster, then federating those instances to a master aggregator. I'm not clear yet on the ramifications of having a proliferation of instances - one per namespace, or even one per deployment type...
The 30 day retention is also an active topic. Eventually a time-series db will be included in the mix for longer term storage (the initial target is a year), but the may need to be some aggregation/retention rules.
> I have not modified the "relabel" section either in the template from that of Syndesis. Take look and modify and add whatever you need to complete the example with instructions, so that we can showcase this example as a demo.
We need to validate that our pods at host:8778/metrics are exposing the metrics https://github.com/teiid/teiid-openshift-examples/blob/0a4383e7c9c3514532...
Based upon that we can change the https://github.com/teiid/teiid-openshift-examples/blob/0a4383e7c9c3514532... relabel line.
> so that we can showcase this example as a demo
The next thing will be some grafana dashboard. The general approach it seems is to keep things as focused as possible (3-4 metrics per board), and a small number of metrics overall - as too much can overwhelm the 30 day retention schedule.
I'm not sure yet about the expectation for grafana instances or the delivery for dashboard definitions.
> Document image generation options
> ---------------------------------
>
> Key: TEIIDSB-23
> URL: https://issues.jboss.org/browse/TEIIDSB-23
> Project: Teiid Spring Boot
> Issue Type: Task
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: Q119
>
>
> We need to document / validate all relevant image options:
> - inclusion of agent bond or other mechanism for jmx exposure to prometheus. There may also be related service annotations
> - annotations for 3scale for rest and openapi exposure of odata
> - any common config options - disk buffer memory, max active plans / engine threads / connection pool sizes. Ideally the buffer manager heap should auto-configure and we should probably always use off-heap for the fixed memory buffer.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
5 years, 10 months