[JBoss JIRA] (TEIID-5265) AssertionError: Batch not found in storage
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5265?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-5265.
-----------------------------------
Resolution: Done
Marking as fixed for now as it's possible the commit addressed this scenario. If there is confirmation of a fix, I'll pull this back to the fix releases. If not, then please reopen the issue.
> AssertionError: Batch not found in storage
> ------------------------------------------
>
> Key: TEIID-5265
> URL: https://issues.jboss.org/browse/TEIID-5265
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 10.1
> Reporter: Bram Gadeyne
> Assignee: Steven Hawkins
> Fix For: 10.2
>
>
> While executing the following query:
> {code:sql}
> insert into rep149_cci(PatientID,age,Orig,Upd,agepointsO,agepointsU,AMI,CHF,PVD,CVD,DEM,CPD,PUD,MLD,SLD,DNC,DWC,HEM,RED,MAL,MST,HIV)
> select cci.patientid as patientid,cci.age as age,cci.orig as orig,cci.upd as upd,cci.agepointso as agepointso,cci.agepointsu as agepointsu,
> cci.charlson_ami as ami,cci.charlson_chf as chf,cci.charlson_pvd as pvd,cci.charlson_cvd as cvd,cci.charlson_dem as dem,cci.charlson_cpd as cpd,cci.charlson_pud as pud,
> cci.charlson_mld as mld,cci.charlson_sld as sld,cci.charlson_dnc as dnc,cci.charlson_dwc as dwc,cci.charlson_hem as hem,cci.charlson_red as red,cci.charlson_mal as mal,
> cci.charlson_mst as mst,cci.charlson_hiv as hiv
> from rep149_cohort gd
> join izisviews.pr_calculate_cci cci on
> cci.patientid_in = gd.patientid;
> {code}
> rep149_cci is defined as
> {code:sql}
> create foreign temporary table rep149_cci(
> PatientID integer not null,
> age bigdecimal,
> Orig long,
> Upd long,
> agepointsO integer,
> agepointsU integer,
> AMI integer,
> CHF integer,
> PVD integer,
> CVD integer,
> DEM integer,
> CPD integer,
> PUD integer,
> MLD integer,
> SLD integer,
> DNC integer,
> DWC integer,
> HEM integer,
> RED integer,
> MAL integer,
> MST integer,
> HIV integer,
> primary key(patientid)
> )OPTIONS (UPDATABLE true) on tmptables;
> {code}
> pr_calculate_cci is a virtual procedure. (I can't however share this code for confidentiality reasons).
> {code:sql}
> CREATE VIRTUAL PROCEDURE pr_calculate_cci(patientid_in integer)
> RETURNS (
> PatientID integer,
> age bigdecimal,
> Orig long,
> Upd long,
> agepointsO integer,
> agepointsU integer,
> AMI integer,
> CHF integer,
> PherVascDis integer,
> CerebVascDis integer,
> Dement integer,
> ChrPulmDis integer,
> PeptUlcer integer,
> MildLivDis integer,
> SevLivDis integer,
> DiabetesNoCompl integer,
> DiabetesWithCompl integer,
> HemiPara integer,
> RenalDis integer,
> Malign integer,
> Metastase integer,
> HIV_AIDS integer
> )
> AS
> BEGIN
> ...
> END
> {code}
> There are a few error reports like this one:
> 2018-02-23 08:33:05,478 ERROR [org.teiid.BUFFER_MGR] (Worker291_QueryProcessorQueue571476) mh0bsTIbs7qY TEIID30001 Max block number exceeded by object 1,366,497 from group 5,068,016. It is a Batch of 1797 rows of [Ljava.lang.String;@10953478. The estimated size was 172,528, but the actual size was 14,410. Increase the maxStorageObjectSize to support larger storage objects. Alternatively you could make the processor batch size smaller.
> The final stack trace is:
> 2018-02-23 08:33:05,539 ERROR [org.teiid.PROCESSOR] (Worker291_QueryProcessorQueue571476) mh0bsTIbs7qY TEIID30019 Unexpected exception for request mh0bsTIbs7qY.90: java.lang.AssertionError: Batch not found in storage 5067771
> at org.teiid.common.buffer.impl.BufferManagerImpl$BatchManagerImpl.getBatch(BufferManagerImpl.java:382)
> at org.teiid.common.buffer.SPage.getValues(SPage.java:234)
> at org.teiid.common.buffer.SPage.search(SPage.java:135)
> at org.teiid.common.buffer.STree.find(STree.java:247)
> at org.teiid.common.buffer.STree.insert(STree.java:295)
> at org.teiid.query.processor.relational.WindowFunctionProjectNode.saveValues(WindowFunctionProjectNode.java:419)
> at org.teiid.query.processor.relational.WindowFunctionProjectNode.buildResults(WindowFunctionProjectNode.java:380)
> at org.teiid.query.processor.relational.WindowFunctionProjectNode.nextBatchDirect(WindowFunctionProjectNode.java:229)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.UnionAllNode.nextBatchDirectInternal(UnionAllNode.java:165)
> at org.teiid.query.processor.relational.UnionAllNode.nextBatchDirect(UnionAllNode.java:138)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.relational.SelectNode.nextBatchDirect(SelectNode.java:100)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
> at org.teiid.query.processor.relational.SourceState.prefetch(SourceState.java:207)
> at org.teiid.query.processor.relational.SourceState.rowCountLE(SourceState.java:154)
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 9 months
[JBoss JIRA] (TEIID-3977) Add support for EnableDependentJoins for Impala data sources
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3977?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-3977:
---------------------------------------
Pulling back to 10.x. There isn't clarity yet on how this would be implemented for impala given temporary tables are not yet supported.
> Add support for EnableDependentJoins for Impala data sources
> -------------------------------------------------------------
>
> Key: TEIID-3977
> URL: https://issues.jboss.org/browse/TEIID-3977
> Project: Teiid
> Issue Type: Feature Request
> Components: JDBC Connector
> Affects Versions: 8.7, 8.12
> Reporter: Anu Saji
> Assignee: Steven Hawkins
> Fix For: 10.x
>
>
> Hi,
> Request to Add support for EnableDependentJoins for Impala data sources. Current docs only suggest [0] [1] Table 13.8
> [0]
> For sources that support temporary tables (DB2, Derby, H2, HSQL 2.0+, MySQL 5.0+, Oracle, PostgreSQL, SQLServer, Sybase) allow dependent join pushdown.
> [1]
> https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Data_Virtuali...
> Thanks
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 9 months
[JBoss JIRA] (TEIID-3977) Add support for EnableDependentJoins for Impala data sources
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3977?page=com.atlassian.jira.plugin... ]
Work on TEIID-3977 stopped by Steven Hawkins.
---------------------------------------------
> Add support for EnableDependentJoins for Impala data sources
> -------------------------------------------------------------
>
> Key: TEIID-3977
> URL: https://issues.jboss.org/browse/TEIID-3977
> Project: Teiid
> Issue Type: Feature Request
> Components: JDBC Connector
> Affects Versions: 8.7, 8.12
> Reporter: Anu Saji
> Assignee: Steven Hawkins
> Fix For: 10.x
>
>
> Hi,
> Request to Add support for EnableDependentJoins for Impala data sources. Current docs only suggest [0] [1] Table 13.8
> [0]
> For sources that support temporary tables (DB2, Derby, H2, HSQL 2.0+, MySQL 5.0+, Oracle, PostgreSQL, SQLServer, Sybase) allow dependent join pushdown.
> [1]
> https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Data_Virtuali...
> Thanks
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 9 months
[JBoss JIRA] (TEIID-3977) Add support for EnableDependentJoins for Impala data sources
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3977?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-3977:
----------------------------------
Fix Version/s: 10.x
(was: 10.2)
> Add support for EnableDependentJoins for Impala data sources
> -------------------------------------------------------------
>
> Key: TEIID-3977
> URL: https://issues.jboss.org/browse/TEIID-3977
> Project: Teiid
> Issue Type: Feature Request
> Components: JDBC Connector
> Affects Versions: 8.7, 8.12
> Reporter: Anu Saji
> Assignee: Steven Hawkins
> Fix For: 10.x
>
>
> Hi,
> Request to Add support for EnableDependentJoins for Impala data sources. Current docs only suggest [0] [1] Table 13.8
> [0]
> For sources that support temporary tables (DB2, Derby, H2, HSQL 2.0+, MySQL 5.0+, Oracle, PostgreSQL, SQLServer, Sybase) allow dependent join pushdown.
> [1]
> https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Data_Virtuali...
> Thanks
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 9 months
[JBoss JIRA] (TEIID-5285) Add high-level feature for redirection of updates
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5285?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-5285:
----------------------------------
Fix Version/s: 10.3
> Add high-level feature for redirection of updates
> -------------------------------------------------
>
> Key: TEIID-5285
> URL: https://issues.jboss.org/browse/TEIID-5285
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine, Teiid Spring Boot
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.3
>
>
> In microservices testing it is desirable to test against production/live data but not commit any updates. We should offer a simple solution that can defined by extension metadata and enabled/disabled by a feature flag. We may need to make simplifying assumptions about the scope (per session, per application, etc.) and durability of the updates.
> Under the covers this will be achieved by using views, update triggers, and a store for the updates and when not enabled the expectation is that all operations should pass through. However the application will be limited to using Teiid SQL and will be required to use the Teiid or pg driver, or Teiid spring boot.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 9 months
[JBoss JIRA] (TEIID-5290) NOW() function returns the same value for each call in BEGIN...END block
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5290?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-5290:
----------------------------------
Fix Version/s: 10.2
Priority: Major (was: Blocker)
> NOW() function returns the same value for each call in BEGIN...END block
> ------------------------------------------------------------------------
>
> Key: TEIID-5290
> URL: https://issues.jboss.org/browse/TEIID-5290
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 10.1
> Environment: teiid-10.1.0 on WildFly Full 11.0.0.Final (WildFly Core 3.0.8.Final)
> Reporter: dalex dalex
> Assignee: Steven Hawkins
> Fix For: 10.2
>
>
> Run the following queries:
> {code:sql}
> BEGIN
> declare timestamp ts1 = select now();
> DECLARE integer c = 20000;
> WHILE (c > 0)
> BEGIN
> c= c-1;
> END
> declare timestamp ts2 = select now();
> select ts1, ts2;
> END ;;
> {code}
> and
> {code:sql}
> BEGIN
> DECLARE integer c = 20000;
> create local temporary table #tem(ts timestamp);
> WHILE (c > 0)
> BEGIN
> INSERT INTO #tem VALUES((select NOW()));
> c= c-1;
> END
> select * from #tem;
> END ;;
> {code}
> NOW() function returns the same value for each call in BEGIN...END block and it doesn't matter if it is atomic or not.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 9 months
[JBoss JIRA] (TEIID-5290) NOW() function returns the same value for each call in BEGIN...END block
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5290?page=com.atlassian.jira.plugin... ]
Work on TEIID-5290 started by Steven Hawkins.
---------------------------------------------
> NOW() function returns the same value for each call in BEGIN...END block
> ------------------------------------------------------------------------
>
> Key: TEIID-5290
> URL: https://issues.jboss.org/browse/TEIID-5290
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 10.1
> Environment: teiid-10.1.0 on WildFly Full 11.0.0.Final (WildFly Core 3.0.8.Final)
> Reporter: dalex dalex
> Assignee: Steven Hawkins
> Fix For: 10.2
>
>
> Run the following queries:
> {code:sql}
> BEGIN
> declare timestamp ts1 = select now();
> DECLARE integer c = 20000;
> WHILE (c > 0)
> BEGIN
> c= c-1;
> END
> declare timestamp ts2 = select now();
> select ts1, ts2;
> END ;;
> {code}
> and
> {code:sql}
> BEGIN
> DECLARE integer c = 20000;
> create local temporary table #tem(ts timestamp);
> WHILE (c > 0)
> BEGIN
> INSERT INTO #tem VALUES((select NOW()));
> c= c-1;
> END
> select * from #tem;
> END ;;
> {code}
> NOW() function returns the same value for each call in BEGIN...END block and it doesn't matter if it is atomic or not.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 9 months
[JBoss JIRA] (TEIID-5290) NOW() function returns the same value for each call in BEGIN...END block
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5290?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5290:
---------------------------------------
> on MSSQL the getdate() function is treated as a non-deterministic
In the scope of a procedure loop yes, but for a single client statement it does appear to be deterministic.
For pg I see that same behavior:
{code}
create temporary table tem (ts timestamp);
DO $$DECLARE r record;
BEGIN
FOR r IN SELECT * FROM generate_series(1,3)
LOOP
perform pg_sleep(1);
insert into tem values ((select clock_timestamp()));
END LOOP;
END$$;
select * from tem;
{code}
All inserted values have a different timestamp.
> What do you think can we do the same here?
Yes, we can change the behavior in the context of procedure loops to be deterministic only to the statement level and not for the entire loop evaluation.
> NOW() function returns the same value for each call in BEGIN...END block
> ------------------------------------------------------------------------
>
> Key: TEIID-5290
> URL: https://issues.jboss.org/browse/TEIID-5290
> Project: Teiid
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: 10.1
> Environment: teiid-10.1.0 on WildFly Full 11.0.0.Final (WildFly Core 3.0.8.Final)
> Reporter: dalex dalex
> Assignee: Steven Hawkins
> Priority: Blocker
>
> Run the following queries:
> {code:sql}
> BEGIN
> declare timestamp ts1 = select now();
> DECLARE integer c = 20000;
> WHILE (c > 0)
> BEGIN
> c= c-1;
> END
> declare timestamp ts2 = select now();
> select ts1, ts2;
> END ;;
> {code}
> and
> {code:sql}
> BEGIN
> DECLARE integer c = 20000;
> create local temporary table #tem(ts timestamp);
> WHILE (c > 0)
> BEGIN
> INSERT INTO #tem VALUES((select NOW()));
> c= c-1;
> END
> select * from #tem;
> END ;;
> {code}
> NOW() function returns the same value for each call in BEGIN...END block and it doesn't matter if it is atomic or not.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
6 years, 9 months