[JBoss JIRA] (TEIID-2035) DeleteDataSource method on Admin API does not completely delete the Connection Factory
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2035?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2035:
-------------------------------------
Here is another variation of the script that is without the remove per the workaround suggested in the original JIRA. Note we are looking for a solution that does not require a restart.
{code}
/subsystem=resource-adapters/resource-adapter=ws/connection-definitions=wsDS:add(jndi-name=java:/wsDS, class-name=org.teiid.resource.adapter.ws.WSManagedConnectionFactory, enabled=true, use-java-context=true)
/subsystem=resource-adapters/resource-adapter=ws/connection-definitions=wsDS/config-properties=EndPoint:add(value=http://foo.com)
/subsystem=resource-adapters/resource-adapter=ws:activate
/subsystem=resource-adapters/resource-adapter=ws/connection-definitions=wsDS/config-properties=EndPoint:remove
/subsystem=resource-adapters/resource-adapter=ws/connection-definitions=wsDS/config-properties=EndPoint:add(value=http://foo1.com)
{code}
I even tried the undefine-attribute instead of "remove" and "add" in last two lines of the script.
> DeleteDataSource method on Admin API does not completely delete the Connection Factory
> --------------------------------------------------------------------------------------
>
> Key: TEIID-2035
> URL: https://issues.jboss.org/browse/TEIID-2035
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Affects Versions: 8.0
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
>
> ConnectionFactory requires a two step creation process
> 1) create the resource-adaptor section
> 2) connection factory section
> The delete is removing the (2) and not (1). Thus any subsequent creates using the same name fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months
[JBoss JIRA] (TEIID-2584) Add management features to materialization
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2584?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2584:
-------------------------------------
For Example if the DDL of view is like this (note all the OPTION properties with "teiid_rel:MATVIEW_" prefix are being proposed)
{code:lang=SQL}
CREATE VIEW actor (
actor_id integer,
first_name varchar(45) NOT NULL,
last_name varchar(45) NOT NULL,
last_update timestamp NOT NULL
) OPTIONS (MATERIALIZED 'TRUE',
MATERIALIZED_TABLE 'pg.public.mat_actor',
teiid_rel:MATVIEW_STATUS_TABLE 'pg.public.status',
teiid_rel:MATVIEW_STAGING_TABLE 'pg.public.mat_actor_staging'
teiid_rel:MATVIEW_TRUNCATE_SCRIPT 'truncate table pg.public.mat_actor_staging'
teiid_rel:MATVIEW_SWAP_SCRIPT 'ALTER TABLE pg.public.mat_actor RENAME TO pg.public.mat_actor_temp; ALTER TABLE pg.public.mat_actor_staging RENAME TO pg.public.mat_actor; ALTER TABLE pg.public.mat_actor_temp RENAME TO pg.public.mat_actor_staging;')
teiid_rel:MATVIEW_LOAD_SCRIPT 'select * into pg.public.mat_actor from sakila.actor OPTION NOCACHE;')
AS SELECT actor_id, first_name, last_name, last_update from pg."public".actor;
{code}
then based on above metadata I will either execute a anonymous block or create a "load" procedure like
{code}
CREATE VIRTUAL PROCEDURE load()
BEGIN
DECLARE string status;
status = (select state FROM teiid_rel:MATVIEW_STATUS_TABLE WHERE tablename='actor');
IF (status <> 'LOADING' or status <> 'LOADED')
BEGIN
update teiid_rel:MATVIEW_STATUS_TABLE set state = 'LOADING' where tablename = 'sakila.actor';
teiid_rel:MATVIEW_TRUNCATE_SCRIPT
teiid_rel:MATVIEW_LOAD_SCRIPT
teiid_rel:MATVIEW_SWAP_SCRIPT
update teiid_rel:MATVIEW_STATUS_TABLE set state = 'LOADED', valid=true, updated=now(), cardinality = VARIABLES.ROWCOUNT where tablename = 'sakila.actor';
EXCEPTION e
update teiid_rel:MATVIEW_STATUS_TABLE set state = 'FAILED_LOAD', valid=false, updated=now(), cardinality = -1 where tablename = 'sakila.actor';
END
END
{code}
and
{code}
CREATE VIRTUAL PROCEDURE needsLoading(ttl, tblName) returns boolean
BEGIN
DECLARE boolean valid;
DECLARE long last_update;
LOOP ON select valid, last_update FROM teiid_rel:MATVIEW_STATUS_TABLE WHERE tablename='tblName' AS status
BEGIN
IF (status.valid = null )
BEGIN
insert into teiid_rel:MATVIEW_STATUS_TABLE (tablename, valid, state, cardinality, updated) values ('tblName', 'false', 'NEEDS_LOADING', -1, now());
END
ELSE (status.valid = false or (status.valid = true and status.last_update < now()-ttl)
BEGIN
update teiid_rel:MATVIEW_STATUS_TABLE set state = 'NEEDS_LOADING' where tablename = 'sakila.actor';
END
END
END
{code}
Where during the execution check the "needsLoading" if true, then execute the "load"
> Add management features to materialization
> ------------------------------------------
>
> Key: TEIID-2584
> URL: https://issues.jboss.org/browse/TEIID-2584
> Project: Teiid
> Issue Type: Feature Request
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Fix For: 8.5
>
>
> Currently Teiid supports internal and external materialization features. The internal is managed completely by the Teiid query engine along with the infinispan cache.
> External materialization is completely unmanaged and left out to the user to manage it externally. This goals for this feature are unify the materialization logic for internal and external, such that both are managed similarly irrespective of the type of materialization chosen.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (TEIID-2619) Add an option to force executions to complete in a single thread
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2619?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-2619.
-----------------------------------
Resolution: Done
Went with the proactive pulling of results in a dedicated thread that are temporarily buffered, which can then be read concurrently by the engine as needed. Updated the reference as well.
> Add an option to force executions to complete in a single thread
> ----------------------------------------------------------------
>
> Key: TEIID-2619
> URL: https://issues.jboss.org/browse/TEIID-2619
> Project: Teiid
> Issue Type: Feature Request
> Components: Connector API, Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 8.5
>
>
> In some cases drivers use thread local or other logic not conducive to Teiid's execution model where the processing/connector thread may be different for each source operation. It would be useful to ensure completion of the source access in a single thread - which could be either to hold thread with the execution or to proactively process the results so that the thread can be released.
> See also TEIID-1648 TEIID-2428
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (TEIID-1648) Reading CLOB's from Teradata has issues and produces exception when stream is read from a different thread than what the clob was read from the resultset
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-1648?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-1648:
---------------------------------------
Removed the KI from the release notes as we have the translator copyLobs property and the threadbound property TEIID-2619 that can force lobs to read by the calling thread prior to be used in another thread. A further refinement would be targeted at just performing the copyLobs/type correction in the translator thread rather than making the execution threadbound - which will the performance/amount of data pulled/saved for all queries not just those with lobs.
> Reading CLOB's from Teradata has issues and produces exception when stream is read from a different thread than what the clob was read from the resultset
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TEIID-1648
> URL: https://issues.jboss.org/browse/TEIID-1648
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 7.4.1
> Reporter: Van Halbert
> Assignee: Steven Hawkins
> Priority: Critical
> Attachments: JDBCQueryServlet.java
>
>
> The initial problem was seen when running BQT testing against the teradata source. However, when debugging, the exception was occurring on the initial client read of the stream. After reading teradata jdbc driver doc, it talks about how its not thread safe and has issues when different threads are handling blobs/clobs. Which lead me to do a simple client test. 2 tests where run run thru a servlet that obtains the teradata connenction via jndi. Thet are as follows (attached is the sample code after the changes used for #2):
> 1. simple execute/select: execute select intkey, objectvalue from bqt1.smalla, for each row, obtain the clob and read the stream (no exception is seen)
> 2. execute select and read clob in a different thread: a new thread is created, passing in the clob, thread is started and then proceeds to try to read the stream, exceptions are seen
> I'll attach the sample code.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (TEIID-2619) Add an option to force executions to complete in a single thread
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-2619:
-------------------------------------
Summary: Add an option to force executions to complete in a single thread
Key: TEIID-2619
URL: https://issues.jboss.org/browse/TEIID-2619
Project: Teiid
Issue Type: Feature Request
Components: Connector API, Query Engine
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 8.5
In some cases drivers use thread local or other logic not conducive to Teiid's execution model where the processing/connector thread may be different for each source operation. It would be useful to ensure completion of the source access in a single thread - which could be either to hold thread with the execution or to proactively process the results so that the thread can be released.
See also TEIID-1648 TEIID-2428
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (TEIID-2618) Add Example of SAP connection
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-2618?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-2618:
-------------------------------------
there is no SAP specific resource adapter; as you are doing you need to use web-service resource-adapter and there is template for it already. May be can add the "security" stuff to it?
> Add Example of SAP connection
> -----------------------------
>
> Key: TEIID-2618
> URL: https://issues.jboss.org/browse/TEIID-2618
> Project: Teiid
> Issue Type: Enhancement
> Affects Versions: 8.4
> Reporter: Warren Gibson
> Assignee: Steven Hawkins
> Priority: Minor
>
> Add an example of connecting to SAP such as:
> <resource-adapter id="sapDS">
> <module slot="main" id="org.jboss.teiid.resource-adapter.webservice"/>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory" jndi-name="java:/sapDS" enabled="true" use-java-context="true" pool-name="sapDS">
> <config-property name="AuthUserName">
> xxxxxxxxxxx
> </config-property>
> <config-property name="SecurityType">
> HTTPBasic
> </config-property>
> <config-property name="RequestTimeout">
> 240000
> </config-property>
> <config-property name="EndPoint"> https://sapes1.sapdevcenter.com/sap/opu/odata/sap/ZCD204_EPM_DEMO_SRV/
> </config-property>
> <config-property name="AuthPassword">
> xxxxxxxxxxx
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (TEIID-2618) Add Example of SAP connection
by Warren Gibson (JIRA)
Warren Gibson created TEIID-2618:
------------------------------------
Summary: Add Example of SAP connection
Key: TEIID-2618
URL: https://issues.jboss.org/browse/TEIID-2618
Project: Teiid
Issue Type: Enhancement
Affects Versions: 8.4
Reporter: Warren Gibson
Assignee: Steven Hawkins
Priority: Minor
Add an example of connecting to SAP such as:
<resource-adapter id="sapDS">
<module slot="main" id="org.jboss.teiid.resource-adapter.webservice"/>
<connection-definitions>
<connection-definition class-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory" jndi-name="java:/sapDS" enabled="true" use-java-context="true" pool-name="sapDS">
<config-property name="AuthUserName">
xxxxxxxxxxx
</config-property>
<config-property name="SecurityType">
HTTPBasic
</config-property>
<config-property name="RequestTimeout">
240000
</config-property>
<config-property name="EndPoint"> https://sapes1.sapdevcenter.com/sap/opu/odata/sap/ZCD204_EPM_DEMO_SRV/
</config-property>
<config-property name="AuthPassword">
xxxxxxxxxxx
</config-property>
</connection-definition>
</connection-definitions>
</resource-adapter>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (TEIID-2578) add/remove schema elements
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-2578?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-2578:
----------------------------------
Summary: add/remove schema elements (was: Provide API to add schema elements at translator layer)
Fix Version/s: 9.0
(was: 8.5)
Description: Schemas are currently static after load. Modifications can only happen with restarts or new versions. We should allow add/drop at runtime. (was: Currently the translator API is can expose the source schema they represent, however there are no facilities to create the schema elements like
* table
* procedure
the function support is available, but this needs be re-factored under this interface (may be this can be a follow on task)
Using this feature, query engine should be able to ad-hoc create tables or procedures. The motivation for the feature is to provide a more comprehensive materialization feature.)
Rewriting the issues as general add/drop of schema entries.
> add/remove schema elements
> --------------------------
>
> Key: TEIID-2578
> URL: https://issues.jboss.org/browse/TEIID-2578
> Project: Teiid
> Issue Type: Feature Request
> Components: Query Engine
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Fix For: 9.0
>
>
> Schemas are currently static after load. Modifications can only happen with restarts or new versions. We should allow add/drop at runtime.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months
[JBoss JIRA] (TEIID-2575) Hive's Binary Type Handling by Teiid
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-2575?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-2575:
------------------------------------------------
Van Halbert <vhalbert(a)redhat.com> changed the Status of [bug 984852|https://bugzilla.redhat.com/show_bug.cgi?id=984852] from NEW to MODIFIED
> Hive's Binary Type Handling by Teiid
> ------------------------------------
>
> Key: TEIID-2575
> URL: https://issues.jboss.org/browse/TEIID-2575
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 8.4.1
> Reporter: Filip Nguyen
> Assignee: Ramesh Reddy
> Fix For: 8.4.1, 8.5
>
>
> I have simple Hive table (column definition: "OBJECTVALUE BINARY" see [1]). When running a simple query "statement.executeQuery("select OBJECTVALUE from smalla;");" I am getting [2]. When I run it simply from Hive console it works ok.
> [1]
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types
> [2]
> {noformat}
> org.teiid.jdbc.TeiidSQLException: TEIID10076 Invalid conversion from type class org.teiid.core.types.BinaryType with value 'EFBFBDEFBFBDEFBFBD' to type class java.lang.String
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:667)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:63)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:516)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:130)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:37)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:75)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:235)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:271)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.teiid.net.socket.SocketServerConnectionFactory$ShutdownHandler.invoke(SocketServerConnectionFactory.java:102)
> at $Proxy6.read(Unknown Source)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:370)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:525)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:393)
> at org.teiid.jdbc.StatementImpl.executeQuery(StatementImpl.java:327)
> at org.jboss.qe.HiveTranslatorTest.simpleQueryTest(HiveTranslatorTest.java:58)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
> at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:46)
> at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:37)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.teiid.core.types.TransformationException: TEIID10076 Invalid conversion from type class org.teiid.core.types.BinaryType with value 'EFBFBDEFBFBDEFBFBD' to type class java.lang.String
> at org.teiid.core.types.DataTypeManager.transformValue(DataTypeManager.java:891)
> at org.teiid.dqp.internal.process.DataTierTupleSource.correctTypes(DataTierTupleSource.java:186)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:326)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:306)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:278)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:149)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:149)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:112)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:153)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:435)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:320)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:248)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:269)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:214)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 8 months