October 2018 - teiid-issues - Jboss List Archives

[JBoss JIRA] (TEIID-4691) Consider replacing JGroups based custom replication with native

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-4691?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-4691: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > Consider replacing JGroups based custom replication with native > --------------------------------------------------------------- > > Key: TEIID-4691 > URL: https://issues.jboss.org/browse/TEIID-4691 > Project: Teiid > Issue Type: Enhancement > Components: Server > Reporter: Ramesh Reddy > Priority: Major > Fix For: 12.x > > > JGroups is responsible for node to node communication. JGroups FRAG2 chunks up the messages that are sizeable by flow control. If you use TCP it will automatically break up the messages into the MTU. Using Jumbo Frames would be recommended. > Note that this will affect how we do the materialization replication among nodes currently. -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4997) Teiid on/with Spark

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-4997?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-4997: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > Teiid on/with Spark > ------------------- > > Key: TEIID-4997 > URL: https://issues.jboss.org/browse/TEIID-4997 > Project: Teiid > Issue Type: Feature Request > Components: Build/Kits, Query Engine > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Priority: Major > Fix For: 12.x > > > With the availability of Spark on OpenShift, we should provide a cooperative planning/execution mode for Teiid that utilizes the Spark engine. > Roughly this would look like a Teiid master running embedded with the Spark master serving the typical JDBC/ODBC/OData endpoints. On an incoming query the optimizer would choose to process against Spark or to process with Teiid - if processing with Teiid that may still require submitting the job to a worker to avoid burdening the master. Alternatively the Teiid master could run in a separate pod with the additional serialization costs, however initially the remote Spark [JDBC/ODBC layer|https://spark.apache.org/docs/latest/sql-programming-guide.html#dis...] will not be available in the OpenShift effort. > If execution against Spark is chosen, then instead of a typical Teiid processor plan a spark job will be created instead. Initially this could be limited to relational plans, but could be expanded to include procedure language support translated to python, scala, etc. The spark job would represent each source access as a [temporary view|https://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc...] accessing the relevant pushdown query. Ideally this would be executed against a Teiid Embedded instance running in the worker node. If remote this would incur an extra hop and have security considerations. This can be thought of as using Teiid for its virtualization and access layer features. The rest of the processing about the access layers could then be represented as Spark SQL. > For example a Teiid user query of "select * from hdfs.tbl h, oracle.tbl o where h.id = o.id order by h.col" would become the Spark SQL job: > CREATE TEMPORARY VIEW h > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:teiid:vdb", > dbtable "(select col ... from hdfs.tbl)", > fetchSize '1024, > ... > ) > CREATE TEMPORARY VIEW o > USING org.apache.spark.sql.jdbc > OPTIONS ( > url "jdbc:teiid:vdb", > dbtable "(select col ... from oracle.tbl)", > fetchSize '1024, > ... > ) > SELECT * FROM h inner join o on h.id > The challenges/considerations of this are: > * Utilizing embedded with coordinated VDB management. There's the associated issue of driver management as well. > * Translating Teiid SQL to Spark SQL. All Teiid functions, udfs, aggregate functions would need to be made known to Spark. Table function constructs, such as XMLTABLE, TEXTTABLE, etc. could initially just be treated as access layer concerns. Type issues would exist as xml/clob/json would map to string. > * no xa support > * we'd need to provide reasonable values for fetch size, partition information, etc. in the access layer queries. > * We'd have to determine the extent to which federated join optimizations need to be conveyed (dependent join and pushdown) as that would go beyond simply translating to Spark SQL. > * there's a potential to use [global temporary views|http://www.gatorsmile.io/globaltempview/] which is a more convenient way of adding virtualization to Spark. > * Large internal materialization should be re-targeted to Spark or JDG -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5040) Add translator override to change the Google spreadsheet

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-5040?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-5040: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > Add translator override to change the Google spreadsheet > -------------------------------------------------------- > > Key: TEIID-5040 > URL: https://issues.jboss.org/browse/TEIID-5040 > Project: Teiid > Issue Type: Enhancement > Components: Misc. Connectors > Reporter: Debbie Steigner > Assignee: Steven Hawkins > Priority: Major > Fix For: 12.x > > > We would like to add a "sheet name" override to the googlesheet resource adapter. So we can configure multiple google sheets without having to configure multiple resource adaptors (1 for each google sheet.) -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4936) Add to tool to generate Couchbase ddl

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-4936?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-4936: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > Add to tool to generate Couchbase ddl > -------------------------------------- > > Key: TEIID-4936 > URL: https://issues.jboss.org/browse/TEIID-4936 > Project: Teiid > Issue Type: Feature Request > Affects Versions: 9.3 > Reporter: Kylin Soong > Assignee: Steven Hawkins > Priority: Optional > Fix For: 12.x > > > The tools can generate ddl.sql, base one some default properties, like sampleSize, sampleKeyspaces, typedNameList, etc, -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-4992) Add OSDQ set functions

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-4992?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-4992: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > Add OSDQ set functions > ---------------------- > > Key: TEIID-4992 > URL: https://issues.jboss.org/browse/TEIID-4992 > Project: Teiid > Issue Type: Enhancement > Components: Query Engine > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Priority: Major > Fix For: 12.x > > Attachments: SetUtil.java > > > There will be a new osdq core library drop in maven central soon containing additional functions for Teiid to add. -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5348) Debezium Integration

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-5348?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-5348: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > Debezium Integration > -------------------- > > Key: TEIID-5348 > URL: https://issues.jboss.org/browse/TEIID-5348 > Project: Teiid > Issue Type: Feature Request > Components: Query Engine > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Priority: Major > Fix For: 12.x > > > Parent issue for Debezium related issues. -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5344) Fully manage the creation of materialization status tables

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-5344?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-5344: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > Fully manage the creation of materialization status tables > ---------------------------------------------------------- > > Key: TEIID-5344 > URL: https://issues.jboss.org/browse/TEIID-5344 > Project: Teiid > Issue Type: Sub-task > Components: Server > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Priority: Major > Fix For: 12.x > > > Status tables should be created if needed on the deployment of a vdb. -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5338) Drop function drops all functions of the same name

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-5338?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-5338: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > Drop function drops all functions of the same name > -------------------------------------------------- > > Key: TEIID-5338 > URL: https://issues.jboss.org/browse/TEIID-5338 > Project: Teiid > Issue Type: Sub-task > Components: Query Engine > Reporter: Steven Hawkins > Assignee: Steven Hawkins > Priority: Major > Fix For: 12.x > > > Our create / drop function ddl support should allow for a unique name to reference functions by rather than removing all functions of a given name. -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5321) SAP IQ numeric functions corner case differences

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-5321?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-5321: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > SAP IQ numeric functions corner case differences > ------------------------------------------------ > > Key: TEIID-5321 > URL: https://issues.jboss.org/browse/TEIID-5321 > Project: Teiid > Issue Type: Bug > Components: Misc. Connectors > Affects Versions: 8.12.13.6_4 > Reporter: Jan Stastny > Assignee: Steven Hawkins > Priority: Minor > Fix For: 12.x > > > The following discrepancies in return values of numeric functions are caused by SAP IQ's behaviour. > # COT(0) > #* results in OutOfRange error in Teiid > #* returns NULL in SAP IQ > # EXP\(n\) - for n < -15 > #* result is 0 in Teiid > #* result is in SAP IQ > #** 0 for -15 > n >=-745 > #** NULL for n<=-746 -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

[JBoss JIRA] (TEIID-5318) MaterializtionManager probe can be too rapid.

by Steven Hawkins (Jira)

[ https://issues.jboss.org/browse/TEIID-5318?page=com.atlassian.jira.plugin... ] Steven Hawkins updated TEIID-5318: ---------------------------------- Fix Version/s: 12.x (was: 11.x) > MaterializtionManager probe can be too rapid. > --------------------------------------------- > > Key: TEIID-5318 > URL: https://issues.jboss.org/browse/TEIID-5318 > Project: Teiid > Issue Type: Quality Risk > Components: Server > Reporter: Steven Hawkins > Priority: Major > Fix For: 12.x > > > If a ttl load is in progress the probe is repeated every minute - regardless of how much time is remaining. This can happen due to manual loads or in clusters where members start the materialization jobs with some skew. While this is not harmful, it does lead to a lot of logging and queries against the status table which could be avoided. -- This message was sent by Atlassian Jira (v7.12.1#712002)

6 years, 2 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

teiid-issues October 2018