[JBoss JIRA] (TEIID-4967) Couchbase - incorrect data returned from VIEW if one value is NULL
by Juraj Duráni (JIRA)
Juraj Duráni created TEIID-4967:
-----------------------------------
Summary: Couchbase - incorrect data returned from VIEW if one value is NULL
Key: TEIID-4967
URL: https://issues.jboss.org/browse/TEIID-4967
Project: Teiid
Issue Type: Bug
Components: Misc. Connectors
Affects Versions: 8.12.x-6.4
Reporter: Juraj Duráni
Assignee: Steven Hawkins
I have a source model with FOREIGN TABLE and view model with VIEW which extracts subset of columns defined in FOREIGN TABLE.
If one value in document is missing (i.e. from Teiid's point of view it's NULL), Teiid returns incorrect data from VIEW.
In some cases, it returns NULL values for multiple columns (even if there is only one column with NULL).
In other cases it shifts values - value from column c contains value of column c+1, column c+1 contains value of c+2 etc.
For Source model, I tried both NATIVE and DDL metadata.
If you need, I can provide you access to testing environment.
*Query:* SELECT * FROM couchbase_small.smalla WHERE intkey = 5;
*Result*
||documentID||FloatNum||IntKey||BigIntegerValue||StringKey||CharValue||LongNum||type||DoubleNum||ObjectValue||ShortValue||BigDecimalValue||DateValue||BooleanValue||TimestampValue||ByteNum||StringNum||TimeValue||IntNum||
|smalla_5|-19|5|-19|5|0|-19|SmallA|-19|-19|-32763|-19|2000-01-06|true|2000-01-01 00:00:05.0|<null>|-19|05:00:00|-19|
*Query:* SELECT * FROM bqt1.smalla WHERE intkey = 5;
*Result:*
||IntKey||StringKey||IntNum||StringNum||FloatNum||LongNum||DoubleNum||ByteNum||DateValue||TimeValue||TimestampValue||BooleanValue||CharValue||ShortValue||BigIntegerValue||BigDecimalValue||ObjectValue||
|5|5|-19|-19|-19|-19|-19|<null>|<null>|<null>|<null>|true|<null>|<null>|<null>|<null>|<null>|
*Query:* SELECT * FROM bqt1.smalla WHERE intkey = 0;
*Result:* Error: TEIID10074 Invalid Byte format in String: 2000-01-01
*Stack trace:*
{code:plain}
12:55:05,263 WARN [org.teiid.CONNECTOR] (Worker15_QueryProcessorQueue279) Connector worker process failed for atomic-request=5x16wQN8Jjck.15.0.61: org.teiid.core.types.TransformationException: TEIID10074 Invalid Byte format in String: 2000-01-01
at org.teiid.core.types.basic.StringToByteTransform.transformDirect(StringToByteTransform.java:43) [teiid-common-core-8.12.11.6_4-redhat-64-4.jar:8.12.11.6_4-redhat-64-4]
at org.teiid.core.types.Transform.transform(Transform.java:47) [teiid-common-core-8.12.11.6_4-redhat-64-4.jar:8.12.11.6_4-redhat-64-4]
at org.teiid.core.types.DataTypeManager.transformValue(DataTypeManager.java:942) [teiid-common-core-8.12.11.6_4-redhat-64-4.jar:8.12.11.6_4-redhat-64-4]
at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.correctTypes(ConnectorWorkItem.java:593)
at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.handleBatch(ConnectorWorkItem.java:446)
at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.more(ConnectorWorkItem.java:236)
at sun.reflect.GeneratedMethodAccessor88.invoke(Unknown Source) [:1.8.0-internal]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0-internal]
at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0-internal]
at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
at com.sun.proxy.$Proxy45.more(Unknown Source)
at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:309)
at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0-internal]
at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280)
at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0-internal]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0-internal]
at java.lang.Thread.run(Thread.java:744) [rt.jar:1.8.0-internal]
{code}
{code:xml|title=VDB}
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<vdb name="couchbase" version="1">
<model name="Couchbase_small" type="PHYSICAL" visible="false">
<source name="couchbase_small" connection-jndi-name="java:/couchbase_small" translator-name="couchbase"/>
<metadata type="DDL">
SET NAMESPACE 'http://www.teiid.org/translator/couchbase/2017' AS teiid_couchbase;
CREATE FOREIGN TABLE SmallA (
documentID string,
FloatNum integer OPTIONS (NAMEINSOURCE '`FloatNum`'),
IntKey integer PRIMARY KEY OPTIONS (NAMEINSOURCE '`IntKey`'),
BigIntegerValue integer OPTIONS (NAMEINSOURCE '`BigIntegerValue`'),
StringKey string OPTIONS (NAMEINSOURCE '`StringKey`'),
CharValue string OPTIONS (NAMEINSOURCE '`CharValue`'),
LongNum integer OPTIONS (NAMEINSOURCE '`LongNum`'),
type string OPTIONS (NAMEINSOURCE '`type`'),
DoubleNum integer OPTIONS (NAMEINSOURCE '`DoubleNum`'),
ObjectValue string OPTIONS (NAMEINSOURCE '`ObjectValue`'),
ShortValue integer OPTIONS (NAMEINSOURCE '`ShortValue`'),
BigDecimalValue integer OPTIONS (NAMEINSOURCE '`BigDecimalValue`'),
DateValue string OPTIONS (NAMEINSOURCE '`DateValue`'),
BooleanValue boolean OPTIONS (NAMEINSOURCE '`BooleanValue`'),
TimestampValue string OPTIONS (NAMEINSOURCE '`TimestampValue`'),
ByteNum integer OPTIONS (NAMEINSOURCE '`ByteNum`'),
StringNum string OPTIONS (NAMEINSOURCE '`StringNum`'),
TimeValue string OPTIONS (NAMEINSOURCE '`TimeValue`'),
IntNum integer OPTIONS (NAMEINSOURCE '`IntNum`')
) OPTIONS (NAMEINSOURCE '`dvqe_small`', UPDATABLE FALSE, "teiid_couchbase:ISARRAYTABLE" 'false', "teiid_couchbase:NAMEDTYPEPAIR" '`type`:''SmallA''');
</metadata>
</model>
<model name="BQT1" type="VIRTUAL">
<metadata type="DDL">
CREATE VIEW SmallA (
IntKey integer PRIMARY KEY,
StringKey string,
IntNum integer,
StringNum string,
FloatNum float,
LongNum long,
DoubleNum double,
ByteNum byte,
DateValue date,
TimeValue time,
TimestampValue timestamp,
BooleanValue boolean,
CharValue char(1),
ShortValue short,
BigIntegerValue biginteger,
BigDecimalValue bigdecimal,
ObjectValue object)
AS
SELECT
IntKey, StringKey, IntNum, StringNum, convert(FloatNum, float) AS FloatNum, convert(LongNum, long) AS LongNum,
convert(DoubleNum, double) AS DoubleNum, convert(ByteNum, byte) AS ByteNum, convert(DateValue, date) AS DateValue,
convert(TimeValue, time) AS TimeValue, convert(TimestampValue, timestamp) AS TimestampValue, BooleanValue,
convert(CharValue, char) AS CharValue, convert(ShortValue, short) AS ShortValue, convert(BigIntegerValue, biginteger) AS BigIntegerValue,
convert(BigDecimalValue, bigdecimal) AS BigDecimalValue, convert(ObjectValue, object) AS ObjectValue
FROM
Couchbase_small.SmallA;
</metadata>
</model>
</vdb>
{code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 11 months
[JBoss JIRA] (TEIID-4966) Infinite loop when deploying a malformed vdb
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4966?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4966:
---------------------------------------
I'm not seeing infinite loop behavior. If I leave the malformed file in place, the log will report in approximately one minute:
15:39:51,106 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 2) WFLYDS0021: Deployment content /home/shawkins/workspaces/teiid-git/teiid/test-integration/common/target/teiid-10.0.0.Alpha1-SNAPSHOT/standalone/deployments/test-vdb.xml appears to be incomplete and is not progressing toward completion. This content cannot be auto-deployed.
15:39:51,107 WARN [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 2) WFLYDS0014: Scan found content configured for auto-deploy that could not be safely auto-deployed. See details above. Deployment changes will not be processed until all problematic content is either removed or whether to deploy the content or not is indicated via a .dodeploy or .skipdeploy marker file. Problematic deployments are [test-vdb.xml]
15:39:51,114 WARN [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0002: Cannot remove extraneous deployment marker file test-vdb.xml.pending
If I update the file, deployment will complete as expected.
However I do agree that the logging is noisy for an invalid xml for a full minute. I don't see anywhere in our logic where we are instigating a retry, so there may need to be an issue against WildFly.
> Infinite loop when deploying a malformed vdb
> ---------------------------------------------
>
> Key: TEIID-4966
> URL: https://issues.jboss.org/browse/TEIID-4966
> Project: Teiid
> Issue Type: Bug
> Environment: * WildFly 10
> * Teiid Server 9.3.0
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
>
> Deploy a malformed vdb. Example with closing comment (--) but without a open comment (!--).
> The following error start to appear in logs without end.
> Deploying a correctly formed VDB after does not fix the problem.
> Log extract;
> 2017-06-22 17:46:48,971 INFO [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0035: Scan found countryServiceList-vdb.xml which is not well-formed at lineNumber: 3591, columnNumber: 9. Either the file was incompletely copied at the time of the scanning or it is just wrong.
> .....
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 11 months
[JBoss JIRA] (TEIID-4964) AWS S3 Connector
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4964?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-4964:
----------------------------------
Component/s: Misc. Connectors
Fix Version/s: Open To Community
10.x
> AWS S3 Connector
> ----------------
>
> Key: TEIID-4964
> URL: https://issues.jboss.org/browse/TEIID-4964
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Reporter: Chandra Akkinepalli
> Assignee: Steven Hawkins
> Fix For: Open To Community, 10.x
>
>
> I would like to request a feature for a new connector to access files in AWS S3 via JDV and be able to use SQL to query them.
> It would be great to have the ability to use AWS KMS for accessing the SSE-KMS encrypted objects on AWS S3
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 11 months
[JBoss JIRA] (TEIID-4663) Support a more secure block mode for client/server encryption
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-4663?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-4663:
------------------------------------------------
Van Halbert <vhalbert(a)redhat.com> changed the Status of [bug 1451866|https://bugzilla.redhat.com/show_bug.cgi?id=1451866] from MODIFIED to ON_QA
> Support a more secure block mode for client/server encryption
> -------------------------------------------------------------
>
> Key: TEIID-4663
> URL: https://issues.jboss.org/browse/TEIID-4663
> Project: Teiid
> Issue Type: Quality Risk
> Components: JDBC Driver, Server
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 9.0.6, 9.1.2, 9.2, 8.12.x-6.4, 8.12.11.6_3
>
>
> ECB is the current default for the socket transport encryption of secure messages. While this is relatively ok for small messages as we also have a message key acting as a CTR counter to some of the blocks, it does not provide strong security - especially for large data volume scenarios, such as when using larger login payloads or the secure requests option. We should default instead to CBC with an explicit initialization vector.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 11 months
[JBoss JIRA] (TEIID-4961) External Materialized View With State Loaded but 0 Cardinality
by Pedro Inácio (JIRA)
[ https://issues.jboss.org/browse/TEIID-4961?page=com.atlassian.jira.plugin... ]
Pedro Inácio commented on TEIID-4961:
-------------------------------------
Depends on the following ones:
* vodafone_nl_raw
* imsi_raw
* sfr_raw
> External Materialized View With State Loaded but 0 Cardinality
> --------------------------------------------------------------
>
> Key: TEIID-4961
> URL: https://issues.jboss.org/browse/TEIID-4961
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 9.3
> Environment: * Teiid 9.3.0
> * MySQL 5.7.18
> * Wildfly 10
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
> Attachments: server.log
>
>
> *Scenario*: Having several views that depend on others (ex: V1-> V2->V3, where V3 depends on V2 and V2 depends on V1), all with external materialization.
> Although all views LoadState is *LOADED *even after failures due to the dependent ones being loaded, in the end some views have zero (0) of cardinality, when there are actually records. Running the queries by hand return records.
> Log attached.
> Example:
> {panel}
> 2017-06-14 17:14:39,521 INFO [org.teiid.MATVIEWS] (Worker63_QueryProcessorQueue19799) WDpoyHmXlkvS Materialization of view VodafoneNlBaseSource.vodafone_nl_base_source completed. Rows updated = 0 Load Number = 1
> {panel}
> After forcing the view update:
> {panel}
> 2017-06-14 17:23:38,042 INFO [org.teiid.MATVIEWS] (Worker65_QueryProcessorQueue27597) +IbRaNRtrZ1K Materialization of view VodafoneNlBaseSource.vodafone_nl_base_source completed. Rows updated = 1816 Load Number = 2
> {panel}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 11 months
[JBoss JIRA] (TEIID-4966) Infinite loop when deploying a malformed vdb
by Pedro Inácio (JIRA)
[ https://issues.jboss.org/browse/TEIID-4966?page=com.atlassian.jira.plugin... ]
Pedro Inácio commented on TEIID-4966:
-------------------------------------
No. I've done it as usual. Overwrite the already existing file.
> Infinite loop when deploying a malformed vdb
> ---------------------------------------------
>
> Key: TEIID-4966
> URL: https://issues.jboss.org/browse/TEIID-4966
> Project: Teiid
> Issue Type: Bug
> Environment: * WildFly 10
> * Teiid Server 9.3.0
> Reporter: Pedro Inácio
> Assignee: Steven Hawkins
>
> Deploy a malformed vdb. Example with closing comment (--) but without a open comment (!--).
> The following error start to appear in logs without end.
> Deploying a correctly formed VDB after does not fix the problem.
> Log extract;
> 2017-06-22 17:46:48,971 INFO [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0035: Scan found countryServiceList-vdb.xml which is not well-formed at lineNumber: 3591, columnNumber: 9. Either the file was incompletely copied at the time of the scanning or it is just wrong.
> .....
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 11 months
[JBoss JIRA] (TEIID-4307) Infinispan DSL Translator: Can't execute update when foreign table name differs from POJO's simple name
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-4307?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-4307:
------------------------------------------------
David Le Sage <dlesage(a)redhat.com> changed the Status of [bug 1351093|https://bugzilla.redhat.com/show_bug.cgi?id=1351093] from VERIFIED to CLOSED
> Infinispan DSL Translator: Can't execute update when foreign table name differs from POJO's simple name
> -------------------------------------------------------------------------------------------------------
>
> Key: TEIID-4307
> URL: https://issues.jboss.org/browse/TEIID-4307
> Project: Teiid
> Issue Type: Enhancement
> Components: Misc. Connectors
> Affects Versions: 8.12.5
> Reporter: Jan Stastny
> Assignee: Van Halbert
> Fix For: 9.1
>
>
> When using Infinispan DSL translator to query data in a jdg instance using POJO, proto file and marshaller, then there's an issue when the foreign table name in a vdb doesn't match simple name of the POJO class.
> Resource adapter:
> {code:xml}
> <resource-adapter id="jdg-mat2">
> <module slot="main" id="org.jboss.teiid.resource-adapter.infinispan.dsl"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.infinispan.dsl.InfinispanManagedConnectionFactory" jndi-name="java:/jdg-mat2-ds" enabled="true" pool-name="jdg-mat2">
> <config-property name="CacheTypeMap">
> rhel7openjdk1_8_localjdg-mat:org.jboss.qe.jdg.pojo.another.CustomerReport2;customerId
> </config-property>
> <config-property name="ProtobufDefinitionFile">
> /org/jboss/qe/jdg/remote/protobuf/another/CustomerReport2.proto
> </config-property>
> <config-property name="Module">
> org.jboss.qe.jdg.pojos
> </config-property>
> <config-property name="MessageDescriptor">
> org.jboss.qe.jdg.remote.protobuf.another.CustomerReport2
> </config-property>
> <config-property name="RemoteServerList">
> dvqe01.mw.lab.eng.bos.redhat.com:11322
> </config-property>
> <config-property name="MessageMarshallers">
> org.jboss.qe.jdg.pojo.another.CustomerReport2:org.jboss.qe.jdg.remote.marshaller.AnotherCustomerReportMarshaller
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
> {code}
> Source model:
> {code:xml}
> <source name="jdgSource2" translator-name="infinispan-cache-dsl"
> connection-jndi-name="java:/jdg-mat2-ds" />
> <metadata type="DDL"><![CDATA[
> CREATE FOREIGN TABLE Customer_Report2(
> customerId integer not null OPTIONS (SEARCHABLE 'Searchable', NATIVE_TYPE 'int'),
> CONSTRAINT PK_ID PRIMARY KEY(customerId)
> ) OPTIONS (NAMEINSOURCE 'rhel7openjdk1_8_localjdg-mat', UPDATABLE TRUE);]]>
> </metadata>
> </model>
> {code}
> Then performing update query (INSERT, UPDATE, DELETE) causes problem.
> Example query:
> {code:sql}
> INSERT INTO jdgSource2.Customer_Report2 VALUES (18);
> {code}
> results in:
> {code:plain}
> 10:00:04,657 WARN [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue4) Connector worker process failed for atomic-request=2v5rhJOyKrpA.2.0.2: org.teiid.translator.TranslatorException: TEIID21005 Registered Class not found for table Customer_Report2, check table name matches a registered class
> {code}
> If there's a requirement for equality between POJO class simple name and modelled table name, it should be clearly stated in documentation as this is very important limitation. Moreover Java classes and SQL tables tend to have different naming conventions.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 11 months
[JBoss JIRA] (TEIID-4875) Planning issue with multiple aggregate decompositions through a join tree
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-4875?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-4875:
------------------------------------------------
Van Halbert <vhalbert(a)redhat.com> changed the Status of [bug 1451870|https://bugzilla.redhat.com/show_bug.cgi?id=1451870] from MODIFIED to ON_QA
> Planning issue with multiple aggregate decompositions through a join tree
> -------------------------------------------------------------------------
>
> Key: TEIID-4875
> URL: https://issues.jboss.org/browse/TEIID-4875
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 9.3, 8.12.x-6.4, 8.12.11.6_3, 9.2.4
>
>
> The logic for determining other symbols from aggregates will not work correctly when a staged grouping is initially pushed and another staged grouping is then pushed above a join that is above that location (typically due to join predicates that pull in other groups).
> A query such as:
> SELECT max(t3.e4), max(t2.e4) as s FROM pm1.g1 as t1, pm1.g2 as t2, pm1.g3 as t3, pm1.g4 as t4, pm2.g1 as t5 WHERE t1.e1 = t2.e1 and (t2.e2 = t3.e2 and t1.e3 || t2.e3 = t3.e3) and t3.e3 = t4.e3 and t4.e4 = t5.e4
> With capabilities that disable join pushdown will fail with an assertionerror that a staged grouping symbol cannot be found during the initialization of the parent join node.
> java.lang.AssertionError: ASSERTION FAILED: expected reference to be not null
> at org.teiid.core.util.Assertion.failed(Assertion.java:73)
> at org.teiid.core.util.Assertion.isNotNull(Assertion.java:100)
> at org.teiid.core.util.Assertion.isNotNull(Assertion.java:92)
> at org.teiid.query.processor.relational.RelationalNode.getProjectionIndexes(RelationalNode.java:367)
> at org.teiid.query.processor.relational.JoinNode.initialize(JoinNode.java:133)
> at org.teiid.query.processor.relational.RelationalPlan.connectExternal(RelationalPlan.java:96)
> at org.teiid.query.processor.relational.RelationalPlan.connectExternal(RelationalPlan.java:102)
> at org.teiid.query.processor.relational.RelationalPlan.connectExternal(RelationalPlan.java:102)
> at org.teiid.query.processor.relational.RelationalPlan.initialize(RelationalPlan.java:91)
> at org.teiid.query.processor.QueryProcessor.init(QueryProcessor.java:226)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:138)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:114)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:164)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:146)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 11 months