[JBoss JIRA] (TEIID-3996) SAP HANA materialization: MATVIEW_ONERROR_ACTION set to WAIT problem
by Jan Stastny (JIRA)
[ https://issues.jboss.org/browse/TEIID-3996?page=com.atlassian.jira.plugin... ]
Jan Stastny commented on TEIID-3996:
------------------------------------
[~shawkins]
I tested with the fix for TEIID-3983 and got it working.
> SAP HANA materialization: MATVIEW_ONERROR_ACTION set to WAIT problem
> --------------------------------------------------------------------
>
> Key: TEIID-3996
> URL: https://issues.jboss.org/browse/TEIID-3996
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.x
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
>
> When using materialization wit SAP HANA datasource, a problem emerges when "teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT' is defined for the materialized view.
> The query which initiates matview's loading doesn't wait for its completion, thus returns stale data (empty resultset in case of the initial invocation).
> Materialized view definition:
> {code:sql}
> CREATE VIEW external_long_ttl (
> customer_id integer NOT NULL,
> total_amount integer
> ) OPTIONS (
> MATERIALIZED 'TRUE',
> UPDATABLE 'FALSE',
> MATERIALIZED_TABLE 'Source.JSTASTNY.dv_matviews_mat_view',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'Source.JSTASTNY.dv_matviews_statustable',
> "teiid_rel:ON_VDB_START_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_create) SELECT id, vdb_create+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:ON_VDB_DROP_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_drop) SELECT id, vdb_drop+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_LOAD_SCRIPT" 'INSERT INTO dv_matviews_mat_view_stage(customer_id,total_amount) SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'exec Source.native(''truncate table dv_matviews_mat_view_stage'');MERGE INTO dv_matviews_check_table(id,before_load) SELECT id, before_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'exec Source.native(''RENAME TABLE dv_matviews_mat_view_stage TO dv_matviews_mat_view_temp'');exec Source.native(''RENAME TABLE dv_matviews_mat_view TO dv_matviews_mat_view_stage'');exec Source.native(''RENAME TABLE dv_matviews_mat_view_temp TO dv_matviews_mat_view'');MERGE INTO dv_matviews_check_table(id,after_load) SELECT id, after_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT',
> "teiid_rel:MATVIEW_TTL" 20000
> ) AS SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;
> {code}
> Query performed:
> {code:sql}
> SELECT * FROM external_long_ttl;
> {code}
> Reproducer:
> {code:java}
> Statement statement = connection.createStatement();
> // invoke initial load, won't return loaded data even though it should
> ResultSet rs1 = statement.executeQuery("SELECT * FROM external_long_ttl;");
> Assert.assertFalse(rs1.next());
> // wait explicitly for the loading to finish
> Thread.sleep(7000);
> // repeat the query, to get the data after load finished
> ResultSet rs2 = statement.executeQuery("SELECT * FROM external_long_ttl;");
> Assert.assertTrue(rs2.next());
> .
> .
> .
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years
[JBoss JIRA] (TEIID-3983) External Materialization MATVIEW_ONERROR_ACTION WAIT problem
by Jan Stastny (JIRA)
[ https://issues.jboss.org/browse/TEIID-3983?page=com.atlassian.jira.plugin... ]
Jan Stastny edited comment on TEIID-3983 at 2/25/16 4:40 AM:
-------------------------------------------------------------
[~shawkins]
Commit 88fa78d doesn't respect property "teiid_rel:MATVIEW_STATUS_TABLE". There is table name 'status' hardcoded.
See https://github.com/teiid/teiid/blob/master/engine/src/main/java/org/teiid... .
was (Author: jstastny):
Commit 88fa78d doesn't respect property "teiid_rel:MATVIEW_STATUS_TABLE". There is table name 'status' hardcoded.
See https://github.com/teiid/teiid/blob/master/engine/src/main/java/org/teiid... .
> External Materialization MATVIEW_ONERROR_ACTION WAIT problem
> ------------------------------------------------------------
>
> Key: TEIID-3983
> URL: https://issues.jboss.org/browse/TEIID-3983
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.5
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
> Fix For: 9.0, 8.12.5, 8.13.2
>
>
> When using "teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT' option in materialized view definition, there's problem with blocked request.
> The blocked request is the one, that triggered the materialized view's loading (first query on the defined materialized view). After such request, one can observe, that the query execution doesn't end, but hangs.
> Meanwhile while examining the contents of 'status' table in the underlying database, the LoadNumber column's value increases regularly according to the defined ttl. Moreover the 'LOADSTATE' column changes from LOADING to LOADED in similar manner. The materialized table is populated with correct data. From this I assume, materialization is set up correctly.
> During the wait, when logging set to DEBUG, there appears regularly, apart from the logs about refreshing the mat view, this entry:
> {code:plain}
> [32m09:14:52,151 DEBUG [org.teiid.PROCESSOR] (Worker0_QueryProcessorQueue2695) Request Thread 87XzBpSzkyk4.0 with state PROCESSING
> [32m09:14:52,151 DEBUG [org.teiid.PROCESSOR] (Worker0_QueryProcessorQueue2695) Request Thread 87XzBpSzkyk4.0 - processor blocked
> {code}
> The log appears in 'ttl' determined intervals, each time after materialized table loading related entries.
> The thread mentioned is the one created after the original query execution was initiated.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years
[JBoss JIRA] (TEIID-3983) External Materialization MATVIEW_ONERROR_ACTION WAIT problem
by Jan Stastny (JIRA)
[ https://issues.jboss.org/browse/TEIID-3983?page=com.atlassian.jira.plugin... ]
Jan Stastny commented on TEIID-3983:
------------------------------------
Commit 88fa78d doesn't respect property "teiid_rel:MATVIEW_STATUS_TABLE". There is table name 'status' hardcoded.
See https://github.com/teiid/teiid/blob/master/engine/src/main/java/org/teiid... .
> External Materialization MATVIEW_ONERROR_ACTION WAIT problem
> ------------------------------------------------------------
>
> Key: TEIID-3983
> URL: https://issues.jboss.org/browse/TEIID-3983
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.5
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
> Fix For: 9.0, 8.12.5, 8.13.2
>
>
> When using "teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT' option in materialized view definition, there's problem with blocked request.
> The blocked request is the one, that triggered the materialized view's loading (first query on the defined materialized view). After such request, one can observe, that the query execution doesn't end, but hangs.
> Meanwhile while examining the contents of 'status' table in the underlying database, the LoadNumber column's value increases regularly according to the defined ttl. Moreover the 'LOADSTATE' column changes from LOADING to LOADED in similar manner. The materialized table is populated with correct data. From this I assume, materialization is set up correctly.
> During the wait, when logging set to DEBUG, there appears regularly, apart from the logs about refreshing the mat view, this entry:
> {code:plain}
> [32m09:14:52,151 DEBUG [org.teiid.PROCESSOR] (Worker0_QueryProcessorQueue2695) Request Thread 87XzBpSzkyk4.0 with state PROCESSING
> [32m09:14:52,151 DEBUG [org.teiid.PROCESSOR] (Worker0_QueryProcessorQueue2695) Request Thread 87XzBpSzkyk4.0 - processor blocked
> {code}
> The log appears in 'ttl' determined intervals, each time after materialized table loading related entries.
> The thread mentioned is the one created after the original query execution was initiated.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years
[JBoss JIRA] (TEIID-3995) SAP HANA materialization:function loadMatView with argument invalidate set to true problem
by Jan Stastny (JIRA)
[ https://issues.jboss.org/browse/TEIID-3995?page=com.atlassian.jira.plugin... ]
Jan Stastny commented on TEIID-3995:
------------------------------------
VDB is defined as:
{code:plain}
<?xml version="1.0" encoding="UTF-8"?>
<vdb name="saphana" version="1">
<description>For testing of materialized views</description>
<model name="Source">
<source name="Source" translator-name="hana-override"
connection-jndi-name="java:/saphana-ds" />
<metadata type="DDL"><![CDATA[
CREATE FOREIGN TABLE "JSTASTNY.DV_MATVIEWS_CHECK_TABLE" (
ID string(255) NOT NULL OPTIONS (NAMEINSOURCE '"ID"', NATIVE_TYPE 'VARCHAR'),
VDB_CREATE integer NOT NULL DEFAULT '0' OPTIONS (NAMEINSOURCE '"VDB_CREATE"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
VDB_DROP integer NOT NULL DEFAULT '0' OPTIONS (NAMEINSOURCE '"VDB_DROP"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
BEFORE_LOAD integer NOT NULL DEFAULT '0' OPTIONS (NAMEINSOURCE '"BEFORE_LOAD"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
AFTER_LOAD integer NOT NULL DEFAULT '0' OPTIONS (NAMEINSOURCE '"AFTER_LOAD"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
CONSTRAINT "_SYS_TREE_RS_#160979_#0_#P0" PRIMARY KEY(ID)
) OPTIONS (NAMEINSOURCE '"JSTASTNY"."DV_MATVIEWS_CHECK_TABLE"', UPDATABLE TRUE);
CREATE FOREIGN TABLE "JSTASTNY.DV_MATVIEWS_CUSTOMERS" (
ID integer NOT NULL OPTIONS (NAMEINSOURCE '"ID"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
NAME string(255) OPTIONS (NAMEINSOURCE '"NAME"', NATIVE_TYPE 'VARCHAR'),
ADDRESS string(255) OPTIONS (NAMEINSOURCE '"ADDRESS"', NATIVE_TYPE 'VARCHAR'),
CONSTRAINT "_SYS_TREE_RS_#160987_#0_#P0" PRIMARY KEY(ID)
) OPTIONS (NAMEINSOURCE '"JSTASTNY"."DV_MATVIEWS_CUSTOMERS"', UPDATABLE TRUE);
CREATE FOREIGN TABLE "JSTASTNY.DV_MATVIEWS_MAT_VIEW" (
CUSTOMER_ID integer NOT NULL OPTIONS (NAMEINSOURCE '"CUSTOMER_ID"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
TOTAL_AMOUNT integer OPTIONS (NAMEINSOURCE '"TOTAL_AMOUNT"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
CONSTRAINT "_SYS_TREE_RS_#160993_#0_#P0" PRIMARY KEY(CUSTOMER_ID)
) OPTIONS (NAMEINSOURCE '"JSTASTNY"."DV_MATVIEWS_MAT_VIEW"', UPDATABLE TRUE);
CREATE FOREIGN TABLE "JSTASTNY.DV_MATVIEWS_MAT_VIEW_STAGE" (
CUSTOMER_ID integer NOT NULL OPTIONS (NAMEINSOURCE '"CUSTOMER_ID"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
TOTAL_AMOUNT integer OPTIONS (NAMEINSOURCE '"TOTAL_AMOUNT"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
CONSTRAINT "_SYS_TREE_RS_#160998_#0_#P0" PRIMARY KEY(CUSTOMER_ID)
) OPTIONS (NAMEINSOURCE '"JSTASTNY"."DV_MATVIEWS_MAT_VIEW_STAGE"', UPDATABLE TRUE);
CREATE FOREIGN TABLE "JSTASTNY.DV_MATVIEWS_ORDERS" (
ID integer NOT NULL OPTIONS (NAMEINSOURCE '"ID"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
AMOUNT integer OPTIONS (NAMEINSOURCE '"AMOUNT"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
CUSTOMER_ID integer OPTIONS (NAMEINSOURCE '"CUSTOMER_ID"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
CONSTRAINT "_SYS_TREE_RS_#161003_#0_#P0" PRIMARY KEY(ID),
CONSTRAINT FKARBPDQCPCCILHM7T1D3HI3L48 FOREIGN KEY(CUSTOMER_ID) REFERENCES JSTASTNY.DV_MATVIEWS_CUSTOMERS (ID)
) OPTIONS (NAMEINSOURCE '"JSTASTNY"."DV_MATVIEWS_ORDERS"', UPDATABLE TRUE);
CREATE FOREIGN TABLE "JSTASTNY.DV_MATVIEWS_STATUSTABLE" (
VDBNAME string(50) NOT NULL OPTIONS (NAMEINSOURCE '"VDBNAME"', NATIVE_TYPE 'VARCHAR'),
VDBVERSION integer NOT NULL OPTIONS (NAMEINSOURCE '"VDBVERSION"', CHAR_OCTET_LENGTH 4, NATIVE_TYPE 'INTEGER'),
SCHEMANAME string(50) NOT NULL OPTIONS (NAMEINSOURCE '"SCHEMANAME"', NATIVE_TYPE 'VARCHAR'),
Name string(256) NOT NULL OPTIONS (NAMEINSOURCE '"NAME"', NATIVE_TYPE 'VARCHAR'),
TARGETSCHEMANAME string(50) OPTIONS (NAMEINSOURCE '"TARGETSCHEMANAME"', NATIVE_TYPE 'VARCHAR'),
TARGETNAME string(256) OPTIONS (NAMEINSOURCE '"TARGETNAME"', NATIVE_TYPE 'VARCHAR'),
VALID boolean NOT NULL OPTIONS (NAMEINSOURCE '"VALID"', CHAR_OCTET_LENGTH 1, NATIVE_TYPE 'TINYINT'),
LOADSTATE string(25) NOT NULL OPTIONS (NAMEINSOURCE '"LOADSTATE"', NATIVE_TYPE 'VARCHAR'),
CARDINALITY long OPTIONS (NAMEINSOURCE '"CARDINALITY"', CHAR_OCTET_LENGTH 20, NATIVE_TYPE 'BIGINT'),
UPDATED timestamp NOT NULL OPTIONS (NAMEINSOURCE '"UPDATED"', CHAR_OCTET_LENGTH 16, NATIVE_TYPE 'TIMESTAMP'),
LOADNUMBER long NOT NULL OPTIONS (NAMEINSOURCE '"LOADNUMBER"', CHAR_OCTET_LENGTH 20, NATIVE_TYPE 'BIGINT'),
CONSTRAINT "_SYS_TREE_RS_#161009_#0_#P0" PRIMARY KEY(VDBNAME, VDBVERSION, SCHEMANAME, Name)
) OPTIONS (NAMEINSOURCE '"JSTASTNY"."DV_MATVIEWS_STATUSTABLE"', UPDATABLE TRUE);
]]>
</metadata>
</model>
<model name="View" type="VIRTUAL">
<metadata type="DDL"><![CDATA[
CREATE VIEW external_long_ttl (
customer_id integer NOT NULL,
total_amount integer
) OPTIONS (MATERIALIZED 'TRUE', UPDATABLE 'FALSE',
MATERIALIZED_TABLE 'Source.JSTASTNY.dv_matviews_mat_view',
"teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
"teiid_rel:MATVIEW_STATUS_TABLE" 'Source.JSTASTNY.dv_matviews_statustable',
"teiid_rel:ON_VDB_START_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_create) SELECT id, vdb_create+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
"teiid_rel:ON_VDB_DROP_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_drop) SELECT id, vdb_drop+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
"teiid_rel:MATVIEW_LOAD_SCRIPT" 'INSERT INTO dv_matviews_mat_view_stage(customer_id,total_amount) SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;',
"teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'exec Source.native(''truncate table dv_matviews_mat_view_stage'');MERGE INTO dv_matviews_check_table(id,before_load) SELECT id, before_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
"teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'exec Source.native(''RENAME TABLE dv_matviews_mat_view_stage TO dv_matviews_mat_view_temp'');exec Source.native(''RENAME TABLE dv_matviews_mat_view TO dv_matviews_mat_view_stage'');exec Source.native(''RENAME TABLE dv_matviews_mat_view_temp TO dv_matviews_mat_view'');MERGE INTO dv_matviews_check_table(id,after_load) SELECT id, after_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
"teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT',
"teiid_rel:MATVIEW_TTL" 20000)
AS SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;
]]>
</metadata>
</model>
<translator name="hana-override" type="hana">
<property name="SupportsDirectQueryProcedure" value="true" />
</translator>
</vdb>
{code}
The stacktrace I get is:
{code:plain}
08:18:07,640 ERROR [org.teiid.TRANSPORT] (New I/O worker #19) TEIID40113 Unhandled exception, aborting operation: org.teiid.transport.ObjectEncoder$FailedWriteException: org.teiid.core.TeiidRuntimeException: TEIID20001 The modeled datatype integer for column 0 doesn't match the runtime type "org.teiid.core.types.ArrayImpl". Please ensure that the column's modeled datatype matches the expected data.
at org.teiid.transport.ObjectEncoder.handleDownstream(ObjectEncoder.java:136) [teiid-runtime-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.jboss.netty.channel.Channels.write(Channels.java:704) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
at org.jboss.netty.channel.Channels.write(Channels.java:671) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:248) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
at org.teiid.transport.SSLAwareChannelHandler$ObjectChannelImpl.write(SSLAwareChannelHandler.java:93) [teiid-runtime-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.transport.SocketClientInstance.send(SocketClientInstance.java:88) [teiid-runtime-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.transport.ServerWorkItem.sendResult(ServerWorkItem.java:135) [teiid-runtime-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.transport.ServerWorkItem$1.onCompletion(ServerWorkItem.java:105) [teiid-runtime-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:135) [teiid-client-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:40) [teiid-client-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:79) [teiid-client-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.dqp.internal.process.RequestWorkItem.sendResultsIfNeeded(RequestWorkItem.java:992) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.dqp.internal.process.RequestWorkItem$1.flushBatchDirect(RequestWorkItem.java:674) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.query.processor.BatchCollector.flushBatch(BatchCollector.java:223) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:194) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:146) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:470) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:348) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:274) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210) [teiid-engine-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_51]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_51]
Caused by: org.teiid.core.TeiidRuntimeException: TEIID20001 The modeled datatype integer for column 0 doesn't match the runtime type "org.teiid.core.types.ArrayImpl". Please ensure that the column's modeled datatype matches the expected data.
at org.teiid.client.BatchSerializer.writeBatch(BatchSerializer.java:878) [teiid-client-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.client.ResultsMessage.writeExternal(ResultsMessage.java:319) [teiid-client-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1459) [rt.jar:1.8.0_51]
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1430) [rt.jar:1.8.0_51]
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) [rt.jar:1.8.0_51]
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) [rt.jar:1.8.0_51]
at org.teiid.net.socket.Message.writeExternal(Message.java:61) [teiid-client-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1459) [rt.jar:1.8.0_51]
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1430) [rt.jar:1.8.0_51]
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) [rt.jar:1.8.0_51]
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) [rt.jar:1.8.0_51]
at org.teiid.transport.ObjectEncoder.handleDownstream(ObjectEncoder.java:131) [teiid-runtime-8.12.5.redhat-1.jar:8.12.5.redhat-1]
... 25 more
Caused by: java.lang.ClassCastException: org.teiid.core.types.ArrayImpl cannot be cast to java.lang.Integer
at org.teiid.client.BatchSerializer$IntColumnSerializer.writeObject(BatchSerializer.java:563) [teiid-client-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.client.BatchSerializer$ColumnSerializer.writeColumn(BatchSerializer.java:534) [teiid-client-8.12.5.redhat-1.jar:8.12.5.redhat-1]
at org.teiid.client.BatchSerializer.writeBatch(BatchSerializer.java:867) [teiid-client-8.12.5.redhat-1.jar:8.12.5.redhat-1]
... 36 more
{code}
> SAP HANA materialization:function loadMatView with argument invalidate set to true problem
> ------------------------------------------------------------------------------------------
>
> Key: TEIID-3995
> URL: https://issues.jboss.org/browse/TEIID-3995
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.x
> Reporter: Jan Stastny
> Assignee: Steven Hawkins
>
> There appears to be an issue with loadMatView function.
> Specifically when the function is invoked with invalidate=>'true' argument, when it is expected, that invocation of this function invalidates the current contents of the materialized view until the initiated load is completed and new data are populated.
> But when calling this function as described, an error occurs:
> {code:plain}
> TEIID20001 The modeled datatype integer for column 0 doesn't match the runtime type "org.teiid.core.types.ArrayImpl". Please ensure that the column's modeled datatype matches the expected data.
> {code}
> Please note, that there is no column of type array in my schema.
> There is declaration of the view:
> {code:sql}
> CREATE VIEW external_long_ttl (
> customer_id integer NOT NULL,
> total_amount integer
> )
> {code}
> and the query used to load the view:
> {code:sql}
> "teiid_rel:MATVIEW_LOAD_SCRIPT" 'INSERT INTO dv_matviews_mat_view_stage(customer_id,total_amount) SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;',
> {code}
> Complete definition of the view is:
> {code:sql}
> CREATE VIEW external_long_ttl (
> customer_id integer NOT NULL,
> total_amount integer
> ) OPTIONS (
> MATERIALIZED 'TRUE',
> UPDATABLE 'FALSE',
> MATERIALIZED_TABLE 'Source.JSTASTNY.dv_matviews_mat_view',
> "teiid_rel:ALLOW_MATVIEW_MANAGEMENT" 'true',
> "teiid_rel:MATVIEW_STATUS_TABLE" 'Source.JSTASTNY.dv_matviews_statustable',
> "teiid_rel:ON_VDB_START_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_create) SELECT id, vdb_create+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:ON_VDB_DROP_SCRIPT" 'MERGE INTO dv_matviews_check_table(id,vdb_drop) SELECT id, vdb_drop+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_LOAD_SCRIPT" 'INSERT INTO dv_matviews_mat_view_stage(customer_id,total_amount) SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;',
> "teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT" 'exec Source.native(''truncate table dv_matviews_mat_view_stage'');MERGE INTO dv_matviews_check_table(id,before_load) SELECT id, before_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT" 'exec Source.native(''RENAME TABLE dv_matviews_mat_view_stage TO dv_matviews_mat_view_temp'');exec Source.native(''RENAME TABLE dv_matviews_mat_view TO dv_matviews_mat_view_stage'');exec Source.native(''RENAME TABLE dv_matviews_mat_view_temp TO dv_matviews_mat_view'');MERGE INTO dv_matviews_check_table(id,after_load) SELECT id, after_load+1 FROM dv_matviews_check_table WHERE id=''external_long_ttl'';',
> "teiid_rel:MATVIEW_ONERROR_ACTION" 'WAIT',
> "teiid_rel:MATVIEW_TTL" 20000
> ) AS SELECT CONVERT(c.id,integer) AS customer_id, CONVERT(SUM(o.amount),integer) AS total_amount FROM dv_matviews_customers c INNER JOIN dv_matviews_orders o ON c.id = o.customer_id GROUP BY c.id;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years
[JBoss JIRA] (TEIID-3993) Teiid: indexOutOfBoundsException on union + count distinct query
by Ivan Chan (JIRA)
[ https://issues.jboss.org/browse/TEIID-3993?page=com.atlassian.jira.plugin... ]
Ivan Chan commented on TEIID-3993:
----------------------------------
Thanks for looking. I have attached the files.
> Teiid: indexOutOfBoundsException on union + count distinct query
> -----------------------------------------------------------------
>
> Key: TEIID-3993
> URL: https://issues.jboss.org/browse/TEIID-3993
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.3
> Environment: Teiid 8.12.3 + posgtesDB
> Reporter: Ivan Chan
> Assignee: Steven Hawkins
> Attachments: teiid_debug.txt, teiid_stack.txt
>
>
> I ran into IndexOfBoundsException from teiid query optimizer when running the following query:
> select "SugarCRMDataSource_public_sales_location"."country" as "SugarCRMDataSource_public_sales_location_country",
> "FoodmartDataSource_public_store"."store_country" as "FoodmartDataSource_public_store_store_country",
> '1|1' as "JS_Discriminator_COL",
> count(DISTINCT "SugarCRMDataSource_public_sales_fact"."amount") as "CountDistinct_SugarCRMDataSource_public_sales_fact_amount"
> from "SugarCRMDataSource_public"."sales_fact" "SugarCRMDataSource_public_sales_fact"
> inner join "SugarCRMDataSource_public"."sales_location" "SugarCRMDataSource_public_sales_location" on ("SugarCRMDataSource_public_sales_fact"."sales_location_id" = "SugarCRMDataSource_public_sales_location"."id")
> inner join "FoodmartDataSource_public"."store" "FoodmartDataSource_public_store" on ("SugarCRMDataSource_public_sales_location"."state" = "FoodmartDataSource_public_store"."store_state")
> group by "SugarCRMDataSource_public_sales_location"."country", "FoodmartDataSource_public_store"."store_country"
> UNION ALL
> select "SugarCRMDataSource_public_sales_location"."country" as "SugarCRMDataSource_public_sales_location_country",
> NULL as "FoodmartDataSource_public_store_store_country",
> '1|0' as "JS_Discriminator_COL",
> count(DISTINCT "SugarCRMDataSource_public_sales_fact"."amount") as "CountDistinct_SugarCRMDataSource_public_sales_fact_amount"
> from "SugarCRMDataSource_public"."sales_fact" "SugarCRMDataSource_public_sales_fact"
> inner join "SugarCRMDataSource_public"."sales_location" "SugarCRMDataSource_public_sales_location" on ("SugarCRMDataSource_public_sales_fact"."sales_location_id" = "SugarCRMDataSource_public_sales_location"."id")
> inner join "FoodmartDataSource_public"."store" "FoodmartDataSource_public_store" on ("SugarCRMDataSource_public_sales_location"."state" = "FoodmartDataSource_public_store"."store_state")
> group by "SugarCRMDataSource_public_sales_location"."country"
> UNION ALL
> select NULL as "SugarCRMDataSource_public_sales_location_country",
> "FoodmartDataSource_public_store"."store_country" as "FoodmartDataSource_public_store_store_country",
> '0|1' as "JS_Discriminator_COL",
> count(DISTINCT "SugarCRMDataSource_public_sales_fact"."amount") as "CountDistinct_SugarCRMDataSource_public_sales_fact_amount"
> from "SugarCRMDataSource_public"."sales_fact" "SugarCRMDataSource_public_sales_fact"
> inner join "SugarCRMDataSource_public"."sales_location" "SugarCRMDataSource_public_sales_location" on ("SugarCRMDataSource_public_sales_fact"."sales_location_id" = "SugarCRMDataSource_public_sales_location"."id")
> inner join "FoodmartDataSource_public"."store" "FoodmartDataSource_public_store" on ("SugarCRMDataSource_public_sales_location"."state" = "FoodmartDataSource_public_store"."store_state")
> group by "FoodmartDataSource_public_store"."store_country"
> UNION ALL
> select NULL as "SugarCRMDataSource_public_sales_location_country",
> NULL as "FoodmartDataSource_public_store_store_country",
> '0|0' as "JS_Discriminator_COL",
> count(DISTINCT "SugarCRMDataSource_public_sales_fact"."amount") as "CountDistinct_SugarCRMDataSource_public_sales_fact_amount"
> from "SugarCRMDataSource_public"."sales_fact" "SugarCRMDataSource_public_sales_fact"
> inner join "SugarCRMDataSource_public"."sales_location" "SugarCRMDataSource_public_sales_location" on ("SugarCRMDataSource_public_sales_fact"."sales_location_id" = "SugarCRMDataSource_public_sales_location"."id")
> inner join "FoodmartDataSource_public"."store" "FoodmartDataSource_public_store" on ("SugarCRMDataSource_public_sales_location"."state" = "FoodmartDataSource_public_store"."store_state")
> order by "SugarCRMDataSource_public_sales_location_country", "FoodmartDataSource_public_store_store_country"
> limit 1000
> And here is the exception:
> Caused by: java.lang.IndexOutOfBoundsException: Index: 4, Size: 4
> at java.util.ArrayList.rangeCheck(ArrayList.java:635)
> at java.util.ArrayList.get(ArrayList.java:411)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.setColStatEstimates(NewCalculateCostUtil.java:411)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.setCardinalityEstimate(NewCalculateCostUtil.java:313)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.estimateSetOpCost(NewCalculateCostUtil.java:252)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.computeNodeCost(NewCalculateCostUtil.java:204)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.updateCardinality(NewCalculateCostUtil.java:136)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.updateCardinality(NewCalculateCostUtil.java:133)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.updateCardinality(NewCalculateCostUtil.java:133)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.computeCostForTree(NewCalculateCostUtil.java:122)
> at org.teiid.query.optimizer.relational.rules.RuleCalculateCost.execute(RuleCalculateCost.java:50)
> at org.teiid.query.optimizer.relational.RelationalPlanner.executeRules(RelationalPlanner.java:807)
> at org.teiid.query.optimizer.relational.RelationalPlanner.optimize(RelationalPlanner.java:223)
> at org.teiid.query.optimizer.QueryOptimizer.optimizePlan(QueryOptimizer.java:159)
> at org.teiid.dqp.internal.process.Request.generatePlan(Request.java:435)
> at org.teiid.dqp.internal.process.PreparedStatementRequest.generatePlan(PreparedStatementRequest.java:119)
> at org.teiid.dqp.internal.process.Request.processRequest(Request.java:463)
> at org.teiid.dqp.internal.process.PreparedStatementRequest.processRequest(PreparedStatementRequest.java:294)
> at org.teiid.dqp.internal.process.RequestWorkItem.processNew(RequestWorkItem.java:640)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:337)
> ... 22 more
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years
[JBoss JIRA] (TEIID-3993) Teiid: indexOutOfBoundsException on union + count distinct query
by Ivan Chan (JIRA)
[ https://issues.jboss.org/browse/TEIID-3993?page=com.atlassian.jira.plugin... ]
Ivan Chan updated TEIID-3993:
-----------------------------
Attachment: teiid_stack.txt
teiid_debug.txt
> Teiid: indexOutOfBoundsException on union + count distinct query
> -----------------------------------------------------------------
>
> Key: TEIID-3993
> URL: https://issues.jboss.org/browse/TEIID-3993
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 8.12.3
> Environment: Teiid 8.12.3 + posgtesDB
> Reporter: Ivan Chan
> Assignee: Steven Hawkins
> Attachments: teiid_debug.txt, teiid_stack.txt
>
>
> I ran into IndexOfBoundsException from teiid query optimizer when running the following query:
> select "SugarCRMDataSource_public_sales_location"."country" as "SugarCRMDataSource_public_sales_location_country",
> "FoodmartDataSource_public_store"."store_country" as "FoodmartDataSource_public_store_store_country",
> '1|1' as "JS_Discriminator_COL",
> count(DISTINCT "SugarCRMDataSource_public_sales_fact"."amount") as "CountDistinct_SugarCRMDataSource_public_sales_fact_amount"
> from "SugarCRMDataSource_public"."sales_fact" "SugarCRMDataSource_public_sales_fact"
> inner join "SugarCRMDataSource_public"."sales_location" "SugarCRMDataSource_public_sales_location" on ("SugarCRMDataSource_public_sales_fact"."sales_location_id" = "SugarCRMDataSource_public_sales_location"."id")
> inner join "FoodmartDataSource_public"."store" "FoodmartDataSource_public_store" on ("SugarCRMDataSource_public_sales_location"."state" = "FoodmartDataSource_public_store"."store_state")
> group by "SugarCRMDataSource_public_sales_location"."country", "FoodmartDataSource_public_store"."store_country"
> UNION ALL
> select "SugarCRMDataSource_public_sales_location"."country" as "SugarCRMDataSource_public_sales_location_country",
> NULL as "FoodmartDataSource_public_store_store_country",
> '1|0' as "JS_Discriminator_COL",
> count(DISTINCT "SugarCRMDataSource_public_sales_fact"."amount") as "CountDistinct_SugarCRMDataSource_public_sales_fact_amount"
> from "SugarCRMDataSource_public"."sales_fact" "SugarCRMDataSource_public_sales_fact"
> inner join "SugarCRMDataSource_public"."sales_location" "SugarCRMDataSource_public_sales_location" on ("SugarCRMDataSource_public_sales_fact"."sales_location_id" = "SugarCRMDataSource_public_sales_location"."id")
> inner join "FoodmartDataSource_public"."store" "FoodmartDataSource_public_store" on ("SugarCRMDataSource_public_sales_location"."state" = "FoodmartDataSource_public_store"."store_state")
> group by "SugarCRMDataSource_public_sales_location"."country"
> UNION ALL
> select NULL as "SugarCRMDataSource_public_sales_location_country",
> "FoodmartDataSource_public_store"."store_country" as "FoodmartDataSource_public_store_store_country",
> '0|1' as "JS_Discriminator_COL",
> count(DISTINCT "SugarCRMDataSource_public_sales_fact"."amount") as "CountDistinct_SugarCRMDataSource_public_sales_fact_amount"
> from "SugarCRMDataSource_public"."sales_fact" "SugarCRMDataSource_public_sales_fact"
> inner join "SugarCRMDataSource_public"."sales_location" "SugarCRMDataSource_public_sales_location" on ("SugarCRMDataSource_public_sales_fact"."sales_location_id" = "SugarCRMDataSource_public_sales_location"."id")
> inner join "FoodmartDataSource_public"."store" "FoodmartDataSource_public_store" on ("SugarCRMDataSource_public_sales_location"."state" = "FoodmartDataSource_public_store"."store_state")
> group by "FoodmartDataSource_public_store"."store_country"
> UNION ALL
> select NULL as "SugarCRMDataSource_public_sales_location_country",
> NULL as "FoodmartDataSource_public_store_store_country",
> '0|0' as "JS_Discriminator_COL",
> count(DISTINCT "SugarCRMDataSource_public_sales_fact"."amount") as "CountDistinct_SugarCRMDataSource_public_sales_fact_amount"
> from "SugarCRMDataSource_public"."sales_fact" "SugarCRMDataSource_public_sales_fact"
> inner join "SugarCRMDataSource_public"."sales_location" "SugarCRMDataSource_public_sales_location" on ("SugarCRMDataSource_public_sales_fact"."sales_location_id" = "SugarCRMDataSource_public_sales_location"."id")
> inner join "FoodmartDataSource_public"."store" "FoodmartDataSource_public_store" on ("SugarCRMDataSource_public_sales_location"."state" = "FoodmartDataSource_public_store"."store_state")
> order by "SugarCRMDataSource_public_sales_location_country", "FoodmartDataSource_public_store_store_country"
> limit 1000
> And here is the exception:
> Caused by: java.lang.IndexOutOfBoundsException: Index: 4, Size: 4
> at java.util.ArrayList.rangeCheck(ArrayList.java:635)
> at java.util.ArrayList.get(ArrayList.java:411)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.setColStatEstimates(NewCalculateCostUtil.java:411)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.setCardinalityEstimate(NewCalculateCostUtil.java:313)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.estimateSetOpCost(NewCalculateCostUtil.java:252)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.computeNodeCost(NewCalculateCostUtil.java:204)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.updateCardinality(NewCalculateCostUtil.java:136)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.updateCardinality(NewCalculateCostUtil.java:133)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.updateCardinality(NewCalculateCostUtil.java:133)
> at org.teiid.query.optimizer.relational.rules.NewCalculateCostUtil.computeCostForTree(NewCalculateCostUtil.java:122)
> at org.teiid.query.optimizer.relational.rules.RuleCalculateCost.execute(RuleCalculateCost.java:50)
> at org.teiid.query.optimizer.relational.RelationalPlanner.executeRules(RelationalPlanner.java:807)
> at org.teiid.query.optimizer.relational.RelationalPlanner.optimize(RelationalPlanner.java:223)
> at org.teiid.query.optimizer.QueryOptimizer.optimizePlan(QueryOptimizer.java:159)
> at org.teiid.dqp.internal.process.Request.generatePlan(Request.java:435)
> at org.teiid.dqp.internal.process.PreparedStatementRequest.generatePlan(PreparedStatementRequest.java:119)
> at org.teiid.dqp.internal.process.Request.processRequest(Request.java:463)
> at org.teiid.dqp.internal.process.PreparedStatementRequest.processRequest(PreparedStatementRequest.java:294)
> at org.teiid.dqp.internal.process.RequestWorkItem.processNew(RequestWorkItem.java:640)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:337)
> ... 22 more
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years
[JBoss JIRA] (TEIID-4000) TeiidSQLException: Adjusted frame length exceeds 2097152
by Steve Tran (JIRA)
[ https://issues.jboss.org/browse/TEIID-4000?page=com.atlassian.jira.plugin... ]
Steve Tran commented on TEIID-4000:
-----------------------------------
As suggested by Steven, I changed this ticket type from Bug to Enhancement, and lowered the priority to Minor.
> TeiidSQLException: Adjusted frame length exceeds 2097152
> --------------------------------------------------------
>
> Key: TEIID-4000
> URL: https://issues.jboss.org/browse/TEIID-4000
> Project: Teiid
> Issue Type: Enhancement
> Affects Versions: 8.7.1.6_2
> Environment: Red Hat JBoss Data Virtualization 6.2 patched to 6.2.2 on EAP6.4.0 patched to version 6.4.3,
> JBoss Developer Studio 8.1.0GA with Teiid Designer plugin 9.0.3.Final.v20150810-1438-B1157
> 64-bit Windows 7 environment
> Reporter: Steve Tran
> Assignee: Steven Hawkins
> Priority: Minor
>
> I'm running a JDBC batch insert statement with Teiid. Here's the error I'm getting when trying to insert 45000 records.
> {code}
> Exception in thread "main" org.teiid.jdbc.TeiidSQLException: Adjusted frame length exceeds 2097152: 6242128 - discarded
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:532)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:393)
> at org.teiid.jdbc.PreparedStatementImpl.executeBatch(PreparedStatementImpl.java:252)
> at com.imshealth.common.Tuning.JdbcDriver.writeData(JdbcDriver.java:166)
> at com.imshealth.common.Tuning.JdbcDriver.main(JdbcDriver.java:78)
> Caused by: org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 2097152: 6242128 - discarded
> at org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFrameDecoder.java:417)
> at org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.failIfNecessary(LengthFieldBasedFrameDecoder.java:405)
> at org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:320)
> at org.teiid.transport.ObjectDecoder.decode(ObjectDecoder.java:108)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:328)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Here's the Java method that creates the batch. Please ignore the lack of Java best practices. This is just test harness code to reproduce the error.
> {code}
> private static final String INSERT_TEIID = "insert into ...<tableA> values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
> private static final String INSERT_ORACLE = "insert into ...<tableA> values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
> private static void writeData(List<List<Object>> records) throws SQLException {
> PreparedStatement ps = connection.prepareStatement(USE_TEIID? INSERT_TEIID : INSERT_ORACLE)
>
> long start = System.currentTimeMillis();
> System.out.println("Building batch PreparedStatement");
> for(List<Object> o : records) {
> ps.setObject(1, o.get(0));
> ps.setObject(2, o.get(1));
> ps.setObject(3, o.get(2));
> ps.setObject(4, o.get(3));
> ps.setObject(5, o.get(4));
> ps.setObject(6, o.get(5));
> ps.setObject(7, o.get(6));
> ps.setObject(8, o.get(7));
> ps.setObject(9, o.get(8));
> ps.setObject(10, o.get(9));
> ps.setObject(11, o.get(10));
> ps.setObject(12, o.get(11));
> ps.setObject(13, o.get(12));
> ps.setObject(14, o.get(13));
> ps.setObject(15, o.get(14));
> ps.setObject(16, o.get(15));
> ps.setObject(17, o.get(16));
> ps.setString(18, o.get(17).toString().replace("]", ""));
> ps.addBatch();
> }
> System.out.println(String.format("Finished building batch PreparedStatement in %d milliseconds", System.currentTimeMillis() - start));
> start = System.currentTimeMillis();
> System.out.println("Executing batch");
> ps.executeBatch();
> System.out.println(String.format("Batch executed in %d milliseconds", System.currentTimeMillis() - start));
> ps.close();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years
[JBoss JIRA] (TEIID-4000) TeiidSQLException: Adjusted frame length exceeds 2097152
by Steve Tran (JIRA)
[ https://issues.jboss.org/browse/TEIID-4000?page=com.atlassian.jira.plugin... ]
Steve Tran updated TEIID-4000:
------------------------------
Priority: Minor (was: Major)
> TeiidSQLException: Adjusted frame length exceeds 2097152
> --------------------------------------------------------
>
> Key: TEIID-4000
> URL: https://issues.jboss.org/browse/TEIID-4000
> Project: Teiid
> Issue Type: Enhancement
> Affects Versions: 8.7.1.6_2
> Environment: Red Hat JBoss Data Virtualization 6.2 patched to 6.2.2 on EAP6.4.0 patched to version 6.4.3,
> JBoss Developer Studio 8.1.0GA with Teiid Designer plugin 9.0.3.Final.v20150810-1438-B1157
> 64-bit Windows 7 environment
> Reporter: Steve Tran
> Assignee: Steven Hawkins
> Priority: Minor
>
> I'm running a JDBC batch insert statement with Teiid. Here's the error I'm getting when trying to insert 45000 records.
> {code}
> Exception in thread "main" org.teiid.jdbc.TeiidSQLException: Adjusted frame length exceeds 2097152: 6242128 - discarded
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:532)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:393)
> at org.teiid.jdbc.PreparedStatementImpl.executeBatch(PreparedStatementImpl.java:252)
> at com.imshealth.common.Tuning.JdbcDriver.writeData(JdbcDriver.java:166)
> at com.imshealth.common.Tuning.JdbcDriver.main(JdbcDriver.java:78)
> Caused by: org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 2097152: 6242128 - discarded
> at org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFrameDecoder.java:417)
> at org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.failIfNecessary(LengthFieldBasedFrameDecoder.java:405)
> at org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:320)
> at org.teiid.transport.ObjectDecoder.decode(ObjectDecoder.java:108)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:328)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Here's the Java method that creates the batch. Please ignore the lack of Java best practices. This is just test harness code to reproduce the error.
> {code}
> private static final String INSERT_TEIID = "insert into ...<tableA> values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
> private static final String INSERT_ORACLE = "insert into ...<tableA> values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
> private static void writeData(List<List<Object>> records) throws SQLException {
> PreparedStatement ps = connection.prepareStatement(USE_TEIID? INSERT_TEIID : INSERT_ORACLE)
>
> long start = System.currentTimeMillis();
> System.out.println("Building batch PreparedStatement");
> for(List<Object> o : records) {
> ps.setObject(1, o.get(0));
> ps.setObject(2, o.get(1));
> ps.setObject(3, o.get(2));
> ps.setObject(4, o.get(3));
> ps.setObject(5, o.get(4));
> ps.setObject(6, o.get(5));
> ps.setObject(7, o.get(6));
> ps.setObject(8, o.get(7));
> ps.setObject(9, o.get(8));
> ps.setObject(10, o.get(9));
> ps.setObject(11, o.get(10));
> ps.setObject(12, o.get(11));
> ps.setObject(13, o.get(12));
> ps.setObject(14, o.get(13));
> ps.setObject(15, o.get(14));
> ps.setObject(16, o.get(15));
> ps.setObject(17, o.get(16));
> ps.setString(18, o.get(17).toString().replace("]", ""));
> ps.addBatch();
> }
> System.out.println(String.format("Finished building batch PreparedStatement in %d milliseconds", System.currentTimeMillis() - start));
> start = System.currentTimeMillis();
> System.out.println("Executing batch");
> ps.executeBatch();
> System.out.println(String.format("Batch executed in %d milliseconds", System.currentTimeMillis() - start));
> ps.close();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years
[JBoss JIRA] (TEIID-4000) TeiidSQLException: Adjusted frame length exceeds 2097152
by Steve Tran (JIRA)
[ https://issues.jboss.org/browse/TEIID-4000?page=com.atlassian.jira.plugin... ]
Steve Tran updated TEIID-4000:
------------------------------
Issue Type: Enhancement (was: Bug)
> TeiidSQLException: Adjusted frame length exceeds 2097152
> --------------------------------------------------------
>
> Key: TEIID-4000
> URL: https://issues.jboss.org/browse/TEIID-4000
> Project: Teiid
> Issue Type: Enhancement
> Affects Versions: 8.7.1.6_2
> Environment: Red Hat JBoss Data Virtualization 6.2 patched to 6.2.2 on EAP6.4.0 patched to version 6.4.3,
> JBoss Developer Studio 8.1.0GA with Teiid Designer plugin 9.0.3.Final.v20150810-1438-B1157
> 64-bit Windows 7 environment
> Reporter: Steve Tran
> Assignee: Steven Hawkins
>
> I'm running a JDBC batch insert statement with Teiid. Here's the error I'm getting when trying to insert 45000 records.
> {code}
> Exception in thread "main" org.teiid.jdbc.TeiidSQLException: Adjusted frame length exceeds 2097152: 6242128 - discarded
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:532)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:393)
> at org.teiid.jdbc.PreparedStatementImpl.executeBatch(PreparedStatementImpl.java:252)
> at com.imshealth.common.Tuning.JdbcDriver.writeData(JdbcDriver.java:166)
> at com.imshealth.common.Tuning.JdbcDriver.main(JdbcDriver.java:78)
> Caused by: org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 2097152: 6242128 - discarded
> at org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFrameDecoder.java:417)
> at org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.failIfNecessary(LengthFieldBasedFrameDecoder.java:405)
> at org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:320)
> at org.teiid.transport.ObjectDecoder.decode(ObjectDecoder.java:108)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
> at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:328)
> at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Here's the Java method that creates the batch. Please ignore the lack of Java best practices. This is just test harness code to reproduce the error.
> {code}
> private static final String INSERT_TEIID = "insert into ...<tableA> values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
> private static final String INSERT_ORACLE = "insert into ...<tableA> values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
> private static void writeData(List<List<Object>> records) throws SQLException {
> PreparedStatement ps = connection.prepareStatement(USE_TEIID? INSERT_TEIID : INSERT_ORACLE)
>
> long start = System.currentTimeMillis();
> System.out.println("Building batch PreparedStatement");
> for(List<Object> o : records) {
> ps.setObject(1, o.get(0));
> ps.setObject(2, o.get(1));
> ps.setObject(3, o.get(2));
> ps.setObject(4, o.get(3));
> ps.setObject(5, o.get(4));
> ps.setObject(6, o.get(5));
> ps.setObject(7, o.get(6));
> ps.setObject(8, o.get(7));
> ps.setObject(9, o.get(8));
> ps.setObject(10, o.get(9));
> ps.setObject(11, o.get(10));
> ps.setObject(12, o.get(11));
> ps.setObject(13, o.get(12));
> ps.setObject(14, o.get(13));
> ps.setObject(15, o.get(14));
> ps.setObject(16, o.get(15));
> ps.setObject(17, o.get(16));
> ps.setString(18, o.get(17).toString().replace("]", ""));
> ps.addBatch();
> }
> System.out.println(String.format("Finished building batch PreparedStatement in %d milliseconds", System.currentTimeMillis() - start));
> start = System.currentTimeMillis();
> System.out.println("Executing batch");
> ps.executeBatch();
> System.out.println(String.format("Batch executed in %d milliseconds", System.currentTimeMillis() - start));
> ps.close();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
9 years