[JBoss JIRA] (TEIID-5969) The "already removed" exception in case of copyLobs=true translator property for MSSQL
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5969?page=com.atlassian.jira.plugi... ]
Steven Hawkins resolved TEIID-5969.
-----------------------------------
Fix Version/s: 13.1.2
14.0.1
Resolution: Done
The cleanup of the lob store associated with the connector work item was too proactive. We either need to mark the item as needing explicit close or let the lob cleanup happen implicitly later. Since we have improved the file store cleanup with other logic it makes the most sense to just allow for the implicit cleanup.
> The "already removed" exception in case of copyLobs=true translator property for MSSQL
> --------------------------------------------------------------------------------------
>
> Key: TEIID-5969
> URL: https://issues.redhat.com/browse/TEIID-5969
> Project: Teiid
> Issue Type: Quality Risk
> Components: Query Engine
> Affects Versions: 13.1
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0, 13.1.2, 14.0.1
>
> Original Estimate: 2 hours
> Time Spent: 2 hours
> Remaining Estimate: 0 minutes
>
> We're getting the following exception:
> {code:java}
> 2020-06-10 16:01:45,326 WARN [org.teiid.PROCESSOR] (Worker21_QueryProcessorQueue272) 1bxj3NJqJLpO TEIID30020 Processing exception for request 1bxj3NJqJLpO.10 'TEIID30328 Unable to evaluate convert(ms_dwh.v1.expr1, string): TEIID30384 Error while evaluating function convert'. Originally ExpressionEvaluationException 'already removed' FileStore.java:156. Enable more detailed logging to see the entire stacktrace. {code}
> in the case of MSSQL connector and copyLobs=true in its translator property.
> After debugging I discovered that the error occurred actually because of not enough value in
> MAX_LOB_MEMORY_BYTES which is used in LobManager. The variable depends on
> org.teiid.maxStringLength which is set usually to 4000 by default. But if someone put a data with text field type in MSSQL having more than, say, 20000 symbols then the field won't be read by Teiid at all, Teiid will just throw out the exception mentioned above. Will it be possible to improve LobManager so that it can read Text data dynamically increasing needed memory for the process? Otherwise even setting the org.teiid.maxStringLength to 20000 Teiid can raise the exception because of a Text data with length more than 20000 symbols.
>
> Steps to reproduce:
> # create the v1 table in MSSQL:
> {code:java}
> CREATE TABLE [dbo].[v1](CREATE TABLE [dbo].[v1]( [expr1] [text] NULL, [b] [int] NULL) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] {code}
> 2. insert in the table the following data (I did it via standard MSSQL Server Management Studio):
> {code:java}
> insert into test_dwh.dbo.v1 values (replicate(convert(varchar(max),'a'), 20000), 1) ; {code}
> 3. add MSSQL as a source to Teiid:
> {code:java}
> <datasource jndi-name="java:/mssql-test-dwh" pool-name="mssql-test-dwh" enabled="true" use-java-context="true">
> <connection-url>jdbc:sqlserver://localhost:1433;databaseName=test_dwh</connection-url>
> <driver-class&gt;com.microsoft.sqlserver.jdbc.SQLServerDriver</driver-class&gt;
> <driver>com.microsoft.sqlserver</driver>
> <new-connection-sql>SET ANSI_WARNINGS OFF</new-connection-sql>
> <pool>
> <min-pool-size>2</min-pool-size>
> <max-pool-size>70</max-pool-size>
> </pool>
> <security>
> <user-name>*****</user-name>
> <password>*****</password>
> </security>
> <validation>
> <check-valid-connection-sql>select 1</check-valid-connection-sql>
> </validation>
> <timeout>
> <blocking-timeout-millis>120000</blocking-timeout-millis>
> <idle-timeout-minutes>5</idle-timeout-minutes>
> </timeout>
> </datasource>{code}
> 4. add ms model:
> {code:java}
> <model name="ms_dwh">
> <property name="importer.useFullSchemaName" value="false"/>
> <property name="importer.tableTypes" value="TABLE,VIEW"/>
> <property name="importer.importKeys" value="false"/>
> <property name="importer.schemaPattern" value="dbo"/>
> <source name="dwh_ms" translator-name="mySqlserver" connection-jndi-name="java:/mssql-test-dwh"/>
> </model>{code}
> 5. add mySqlserver translator:
> {code:java}
> <translator name="mySqlserver" type="sqlserver">
> <property name="SupportsNativeQueries" value="true"/>
> <property name="copyLobs" value="true"/>
> </translator> {code}
> 6. try to run the following query:
> {code:java}
> select cast(expr1 as string) from ms_dwh.v1 ;; {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5969) The "already removed" exception in case of copyLobs=true translator property for MSSQL
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5969?page=com.atlassian.jira.plugi... ]
Steven Hawkins updated TEIID-5969:
----------------------------------
Original Estimate: 2 hours
Remaining Estimate: 2 hours
Story Points: 0.5
Sprint: DV Sprint 65
> The "already removed" exception in case of copyLobs=true translator property for MSSQL
> --------------------------------------------------------------------------------------
>
> Key: TEIID-5969
> URL: https://issues.redhat.com/browse/TEIID-5969
> Project: Teiid
> Issue Type: Quality Risk
> Components: Query Engine
> Affects Versions: 13.1
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 2 hours
> Remaining Estimate: 2 hours
>
> We're getting the following exception:
> {code:java}
> 2020-06-10 16:01:45,326 WARN [org.teiid.PROCESSOR] (Worker21_QueryProcessorQueue272) 1bxj3NJqJLpO TEIID30020 Processing exception for request 1bxj3NJqJLpO.10 'TEIID30328 Unable to evaluate convert(ms_dwh.v1.expr1, string): TEIID30384 Error while evaluating function convert'. Originally ExpressionEvaluationException 'already removed' FileStore.java:156. Enable more detailed logging to see the entire stacktrace. {code}
> in the case of MSSQL connector and copyLobs=true in its translator property.
> After debugging I discovered that the error occurred actually because of not enough value in
> MAX_LOB_MEMORY_BYTES which is used in LobManager. The variable depends on
> org.teiid.maxStringLength which is set usually to 4000 by default. But if someone put a data with text field type in MSSQL having more than, say, 20000 symbols then the field won't be read by Teiid at all, Teiid will just throw out the exception mentioned above. Will it be possible to improve LobManager so that it can read Text data dynamically increasing needed memory for the process? Otherwise even setting the org.teiid.maxStringLength to 20000 Teiid can raise the exception because of a Text data with length more than 20000 symbols.
>
> Steps to reproduce:
> # create the v1 table in MSSQL:
> {code:java}
> CREATE TABLE [dbo].[v1](CREATE TABLE [dbo].[v1]( [expr1] [text] NULL, [b] [int] NULL) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] {code}
> 2. insert in the table the following data (I did it via standard MSSQL Server Management Studio):
> {code:java}
> insert into test_dwh.dbo.v1 values (replicate(convert(varchar(max),'a'), 20000), 1) ; {code}
> 3. add MSSQL as a source to Teiid:
> {code:java}
> <datasource jndi-name="java:/mssql-test-dwh" pool-name="mssql-test-dwh" enabled="true" use-java-context="true">
> <connection-url>jdbc:sqlserver://localhost:1433;databaseName=test_dwh</connection-url>
> <driver-class&gt;com.microsoft.sqlserver.jdbc.SQLServerDriver</driver-class&gt;
> <driver>com.microsoft.sqlserver</driver>
> <new-connection-sql>SET ANSI_WARNINGS OFF</new-connection-sql>
> <pool>
> <min-pool-size>2</min-pool-size>
> <max-pool-size>70</max-pool-size>
> </pool>
> <security>
> <user-name>*****</user-name>
> <password>*****</password>
> </security>
> <validation>
> <check-valid-connection-sql>select 1</check-valid-connection-sql>
> </validation>
> <timeout>
> <blocking-timeout-millis>120000</blocking-timeout-millis>
> <idle-timeout-minutes>5</idle-timeout-minutes>
> </timeout>
> </datasource>{code}
> 4. add ms model:
> {code:java}
> <model name="ms_dwh">
> <property name="importer.useFullSchemaName" value="false"/>
> <property name="importer.tableTypes" value="TABLE,VIEW"/>
> <property name="importer.importKeys" value="false"/>
> <property name="importer.schemaPattern" value="dbo"/>
> <source name="dwh_ms" translator-name="mySqlserver" connection-jndi-name="java:/mssql-test-dwh"/>
> </model>{code}
> 5. add mySqlserver translator:
> {code:java}
> <translator name="mySqlserver" type="sqlserver">
> <property name="SupportsNativeQueries" value="true"/>
> <property name="copyLobs" value="true"/>
> </translator> {code}
> 6. try to run the following query:
> {code:java}
> select cast(expr1 as string) from ms_dwh.v1 ;; {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5969) The "already removed" exception in case of copyLobs=true translator property for MSSQL
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5969?focusedWorklogId=12451504&pag... ]
Steven Hawkins logged work on TEIID-5969:
-----------------------------------------
Author: Steven Hawkins
Created on: 16/Jun/20 10:25 AM
Start Date: 16/Jun/20 10:25 AM
Worklog Time Spent: 2 hours
Issue Time Tracking
-------------------
Remaining Estimate: 0 minutes (was: 2 hours)
Time Spent: 2 hours
Worklog Id: (was: 12451504)
> The "already removed" exception in case of copyLobs=true translator property for MSSQL
> --------------------------------------------------------------------------------------
>
> Key: TEIID-5969
> URL: https://issues.redhat.com/browse/TEIID-5969
> Project: Teiid
> Issue Type: Quality Risk
> Components: Query Engine
> Affects Versions: 13.1
> Reporter: Dmitrii Pogorelov
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0
>
> Original Estimate: 2 hours
> Time Spent: 2 hours
> Remaining Estimate: 0 minutes
>
> We're getting the following exception:
> {code:java}
> 2020-06-10 16:01:45,326 WARN [org.teiid.PROCESSOR] (Worker21_QueryProcessorQueue272) 1bxj3NJqJLpO TEIID30020 Processing exception for request 1bxj3NJqJLpO.10 'TEIID30328 Unable to evaluate convert(ms_dwh.v1.expr1, string): TEIID30384 Error while evaluating function convert'. Originally ExpressionEvaluationException 'already removed' FileStore.java:156. Enable more detailed logging to see the entire stacktrace. {code}
> in the case of MSSQL connector and copyLobs=true in its translator property.
> After debugging I discovered that the error occurred actually because of not enough value in
> MAX_LOB_MEMORY_BYTES which is used in LobManager. The variable depends on
> org.teiid.maxStringLength which is set usually to 4000 by default. But if someone put a data with text field type in MSSQL having more than, say, 20000 symbols then the field won't be read by Teiid at all, Teiid will just throw out the exception mentioned above. Will it be possible to improve LobManager so that it can read Text data dynamically increasing needed memory for the process? Otherwise even setting the org.teiid.maxStringLength to 20000 Teiid can raise the exception because of a Text data with length more than 20000 symbols.
>
> Steps to reproduce:
> # create the v1 table in MSSQL:
> {code:java}
> CREATE TABLE [dbo].[v1](CREATE TABLE [dbo].[v1]( [expr1] [text] NULL, [b] [int] NULL) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] {code}
> 2. insert in the table the following data (I did it via standard MSSQL Server Management Studio):
> {code:java}
> insert into test_dwh.dbo.v1 values (replicate(convert(varchar(max),'a'), 20000), 1) ; {code}
> 3. add MSSQL as a source to Teiid:
> {code:java}
> <datasource jndi-name="java:/mssql-test-dwh" pool-name="mssql-test-dwh" enabled="true" use-java-context="true">
> <connection-url>jdbc:sqlserver://localhost:1433;databaseName=test_dwh</connection-url>
> <driver-class&gt;com.microsoft.sqlserver.jdbc.SQLServerDriver</driver-class&gt;
> <driver>com.microsoft.sqlserver</driver>
> <new-connection-sql>SET ANSI_WARNINGS OFF</new-connection-sql>
> <pool>
> <min-pool-size>2</min-pool-size>
> <max-pool-size>70</max-pool-size>
> </pool>
> <security>
> <user-name>*****</user-name>
> <password>*****</password>
> </security>
> <validation>
> <check-valid-connection-sql>select 1</check-valid-connection-sql>
> </validation>
> <timeout>
> <blocking-timeout-millis>120000</blocking-timeout-millis>
> <idle-timeout-minutes>5</idle-timeout-minutes>
> </timeout>
> </datasource>{code}
> 4. add ms model:
> {code:java}
> <model name="ms_dwh">
> <property name="importer.useFullSchemaName" value="false"/>
> <property name="importer.tableTypes" value="TABLE,VIEW"/>
> <property name="importer.importKeys" value="false"/>
> <property name="importer.schemaPattern" value="dbo"/>
> <source name="dwh_ms" translator-name="mySqlserver" connection-jndi-name="java:/mssql-test-dwh"/>
> </model>{code}
> 5. add mySqlserver translator:
> {code:java}
> <translator name="mySqlserver" type="sqlserver">
> <property name="SupportsNativeQueries" value="true"/>
> <property name="copyLobs" value="true"/>
> </translator> {code}
> 6. try to run the following query:
> {code:java}
> select cast(expr1 as string) from ms_dwh.v1 ;; {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5968) enabled=true flag not visible in xa-datasource
by Manoj Majumdar (Jira)
[ https://issues.redhat.com/browse/TEIID-5968?page=com.atlassian.jira.plugi... ]
Manoj Majumdar commented on TEIID-5968:
---------------------------------------
Ok, so is there a way we can get a build for this change for now?
> enabled=true flag not visible in xa-datasource
> ----------------------------------------------
>
> Key: TEIID-5968
> URL: https://issues.redhat.com/browse/TEIID-5968
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0, 13.1.2, 14.0.1
>
> Original Estimate: 2 hours, 30 minutes
> Remaining Estimate: 2 hours, 30 minutes
>
> While creating a XA-Datasource using admin Api, enabled="true" paramter is not set and is not visible in standalone-teiid.xml, have to add it manually.
>
> Because of this flag not set, connection pooling is not working and teiid is creating more connections than max-pool-size mentioned
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5968) enabled=true flag not visible in xa-datasource
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIID-5968?page=com.atlassian.jira.plugi... ]
Steven Hawkins commented on TEIID-5968:
---------------------------------------
A rough schedule is usually given with the release announcements: http://teiid.io/blog/
It will be a couple of weeks before a community release is made on any of those branches.
> enabled=true flag not visible in xa-datasource
> ----------------------------------------------
>
> Key: TEIID-5968
> URL: https://issues.redhat.com/browse/TEIID-5968
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0, 13.1.2, 14.0.1
>
> Original Estimate: 2 hours, 30 minutes
> Remaining Estimate: 2 hours, 30 minutes
>
> While creating a XA-Datasource using admin Api, enabled="true" paramter is not set and is not visible in standalone-teiid.xml, have to add it manually.
>
> Because of this flag not set, connection pooling is not working and teiid is creating more connections than max-pool-size mentioned
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIID-5968) enabled=true flag not visible in xa-datasource
by Manoj Majumdar (Jira)
[ https://issues.redhat.com/browse/TEIID-5968?page=com.atlassian.jira.plugi... ]
Manoj Majumdar commented on TEIID-5968:
---------------------------------------
When will the fixed versions for admin api be released?
> enabled=true flag not visible in xa-datasource
> ----------------------------------------------
>
> Key: TEIID-5968
> URL: https://issues.redhat.com/browse/TEIID-5968
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Affects Versions: 11.1.2
> Reporter: Manoj Majumdar
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 15.0, 13.1.2, 14.0.1
>
> Original Estimate: 2 hours, 30 minutes
> Remaining Estimate: 2 hours, 30 minutes
>
> While creating a XA-Datasource using admin Api, enabled="true" paramter is not set and is not visible in standalone-teiid.xml, have to add it manually.
>
> Because of this flag not set, connection pooling is not working and teiid is creating more connections than max-pool-size mentioned
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIIDSB-209) NPE using salesforce connection
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-209?focusedWorklogId=12451500&pa... ]
Steven Hawkins logged work on TEIIDSB-209:
------------------------------------------
Author: Steven Hawkins
Created on: 15/Jun/20 3:20 PM
Start Date: 15/Jun/20 3:20 PM
Worklog Time Spent: 30 minutes
Issue Time Tracking
-------------------
Remaining Estimate: 0 minutes (was: 30 minutes)
Time Spent: 30 minutes
Worklog Id: (was: 12451500)
> NPE using salesforce connection
> -------------------------------
>
> Key: TEIIDSB-209
> URL: https://issues.redhat.com/browse/TEIIDSB-209
> Project: Teiid Spring Boot
> Issue Type: Bug
> Components: datasource
> Reporter: Steven Hawkins
> Priority: Major
> Fix For: 1.6.0
>
> Original Estimate: 30 minutes
> Time Spent: 30 minutes
> Remaining Estimate: 0 minutes
>
> The config is not being set appropriately and results in an npe. This is due to TEIIDSB-184
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIIDSB-210) Add more javadocs and make configuration consistent
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-210?page=com.atlassian.jira.plug... ]
Steven Hawkins resolved TEIIDSB-210.
------------------------------------
Assignee: Steven Hawkins
Resolution: Done
The javadocs were added on the teiid side. The initial inconsistency addressed was username (vs userName). It looks like we may need to eventually have to ensure consistency in terms like filename and the set of properties related to trust/key stores.
> Add more javadocs and make configuration consistent
> ---------------------------------------------------
>
> Key: TEIIDSB-210
> URL: https://issues.redhat.com/browse/TEIIDSB-210
> Project: Teiid Spring Boot
> Issue Type: Bug
> Components: datasource
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 1.6.0
>
> Original Estimate: 2 hours
> Remaining Estimate: 2 hours
>
> Most of the base configuration classes don't have any javadocs. We should add anything that is meaningful and make sure that property names are consistent.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years
[JBoss JIRA] (TEIIDSB-210) Add more javadocs and make configuration consistent
by Steven Hawkins (Jira)
[ https://issues.redhat.com/browse/TEIIDSB-210?focusedWorklogId=12451499&pa... ]
Steven Hawkins logged work on TEIIDSB-210:
------------------------------------------
Author: Steven Hawkins
Created on: 15/Jun/20 3:19 PM
Start Date: 15/Jun/20 3:19 PM
Worklog Time Spent: 2 hours
Issue Time Tracking
-------------------
Remaining Estimate: 0 minutes (was: 2 hours)
Time Spent: 2 hours
Worklog Id: (was: 12451499)
> Add more javadocs and make configuration consistent
> ---------------------------------------------------
>
> Key: TEIIDSB-210
> URL: https://issues.redhat.com/browse/TEIIDSB-210
> Project: Teiid Spring Boot
> Issue Type: Bug
> Components: datasource
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Priority: Major
> Fix For: 1.6.0
>
> Original Estimate: 2 hours
> Time Spent: 2 hours
> Remaining Estimate: 0 minutes
>
> Most of the base configuration classes don't have any javadocs. We should add anything that is meaningful and make sure that property names are consistent.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years