[JBoss JIRA] (TEIID-4736) JDG resource adapter does not reconnect properly when JDG as mat cache is restarted
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4736?page=com.atlassian.jira.plugin... ]
Van Halbert commented on TEIID-4736:
------------------------------------
The PR for pushing the changes upstream: https://github.com/teiid/teiid/pull/913 (this is also includes TEIID-4709)
> JDG resource adapter does not reconnect properly when JDG as mat cache is restarted
> -----------------------------------------------------------------------------------
>
> Key: TEIID-4736
> URL: https://issues.jboss.org/browse/TEIID-4736
> Project: Teiid
> Issue Type: Sub-task
> Components: JDG Connector
> Affects Versions: 8.12.x, 9.0.1
> Reporter: William DeCoste
> Assignee: Van Halbert
> Fix For: 9.3, 8.12.9.6_3, 9.2.1
>
>
> After JDG restart, JDV cannot recover:
> Executing using the TeiidDriver
> org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 PersonMatSource1: Program Error: Cache Name is null
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:721)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:64)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:560)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:135)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:40)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:79)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:268)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:306)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.net.socket.SocketServerConnectionFactory$ShutdownHandler.invoke(SocketServerConnectionFactory.java:98)
> at com.sun.proxy.$Proxy24.read(Unknown Source)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:405)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:569)
> at org.teiid.jdbc.StatementImpl.execute(StatementImpl.java:1063)
> at org.teiid.jdbc.StatementImpl.execute(StatementImpl.java:338)
> at JDBCClient.execute(JDBCClient.java:93)
> at JDBCClient.main(JDBCClient.java:48)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 PersonMatSource1: Program Error: Cache Name is null
> at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:401)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:161)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:391)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:145)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:151)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:114)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:164)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:146)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:472)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:348)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:274)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ... 1 more
> Caused by: org.teiid.core.TeiidException: Remote org.teiid.translator.TranslatorException: Program Error: Cache Name is null
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.handleError(ConnectorWorkItem.java:310)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:369)
> at sun.reflect.GeneratedMethodAccessor103.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy113.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
> ... 6 more
> Caused by: java.lang.Error: Remote java.lang.AssertionError: Program Error: Cache Name is null
> at org.teiid.core.util.Assertion.failed(Assertion.java:73)
> at org.teiid.core.util.Assertion.isNotNull(Assertion.java:100)
> at org.teiid.resource.adapter.infinispan.dsl.InfinispanManagedConnectionFactory.getCache(InfinispanManagedConnectionFactory.java:173)
> at org.teiid.resource.adapter.infinispan.dsl.InfinispanConnectionImpl.getCache(InfinispanConnectionImpl.java:251)
> at org.teiid.resource.adapter.infinispan.dsl.InfinispanConnectionImpl.getQueryFactory(InfinispanConnectionImpl.java:140)
> at org.teiid.resource.adapter.infinispan.dsl.DSLSearch.getQueryBuilder(DSLSearch.java:205)
> at org.teiid.resource.adapter.infinispan.dsl.DSLSearch.performSearch(DSLSearch.java:156)
> at org.teiid.resource.adapter.infinispan.dsl.DSLSearch.performSearch(DSLSearch.java:143)
> at org.teiid.translator.object.ObjectExecution.execute(ObjectExecution.java:261)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:366)
> ... 16 more
> -----------------------------------
> Executing using the TeiidDataSource
> org.teiid.jdbc.TeiidSQLException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 PersonMatSource1: Program Error: Cache Name is null
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:135)
> at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:71)
> at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:721)
> at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:64)
> at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:560)
> at org.teiid.client.util.ResultsFuture.done(ResultsFuture.java:135)
> at org.teiid.client.util.ResultsFuture.access$200(ResultsFuture.java:40)
> at org.teiid.client.util.ResultsFuture$1.receiveResults(ResultsFuture.java:79)
> at org.teiid.net.socket.SocketServerInstanceImpl.receivedMessage(SocketServerInstanceImpl.java:268)
> at org.teiid.net.socket.SocketServerInstanceImpl.read(SocketServerInstanceImpl.java:306)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.net.socket.SocketServerConnectionFactory$ShutdownHandler.invoke(SocketServerConnectionFactory.java:98)
> at com.sun.proxy.$Proxy24.read(Unknown Source)
> at org.teiid.net.socket.SocketServerInstanceImpl$RemoteInvocationHandler$1.get(SocketServerInstanceImpl.java:405)
> at org.teiid.jdbc.StatementImpl.executeSql(StatementImpl.java:569)
> at org.teiid.jdbc.StatementImpl.execute(StatementImpl.java:1063)
> at org.teiid.jdbc.StatementImpl.execute(StatementImpl.java:338)
> at JDBCClient.execute(JDBCClient.java:93)
> at JDBCClient.main(JDBCClient.java:55)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.teiid.core.TeiidProcessingException: TEIID30504 Remote org.teiid.core.TeiidProcessingException: TEIID30504 PersonMatSource1: Program Error: Cache Name is null
> at org.teiid.dqp.internal.process.DataTierTupleSource.exceptionOccurred(DataTierTupleSource.java:401)
> at org.teiid.dqp.internal.process.DataTierTupleSource.nextTuple(DataTierTupleSource.java:161)
> at org.teiid.query.processor.relational.AccessNode.nextBatchDirect(AccessNode.java:391)
> at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:282)
> at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:145)
> at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:151)
> at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:114)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:164)
> at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:146)
> at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:472)
> at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:348)
> at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:51)
> at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:274)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ... 1 more
> Caused by: org.teiid.core.TeiidException: Remote org.teiid.translator.TranslatorException: Program Error: Cache Name is null
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.handleError(ConnectorWorkItem.java:310)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:369)
> at sun.reflect.GeneratedMethodAccessor103.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy113.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
> ... 6 more
> Caused by: java.lang.Error: Remote java.lang.AssertionError: Program Error: Cache Name is null
> at org.teiid.core.util.Assertion.failed(Assertion.java:73)
> at org.teiid.core.util.Assertion.isNotNull(Assertion.java:100)
> at org.teiid.resource.adapter.infinispan.dsl.InfinispanManagedConnectionFactory.getCache(InfinispanManagedConnectionFactory.java:173)
> at org.teiid.resource.adapter.infinispan.dsl.InfinispanConnectionImpl.getCache(InfinispanConnectionImpl.java:251)
> at org.teiid.resource.adapter.infinispan.dsl.InfinispanConnectionImpl.getQueryFactory(InfinispanConnectionImpl.java:140)
> at org.teiid.resource.adapter.infinispan.dsl.DSLSearch.getQueryBuilder(DSLSearch.java:205)
> at org.teiid.resource.adapter.infinispan.dsl.DSLSearch.performSearch(DSLSearch.java:156)
> at org.teiid.resource.adapter.infinispan.dsl.DSLSearch.performSearch(DSLSearch.java:143)
> at org.teiid.translator.object.ObjectExecution.execute(ObjectExecution.java:261)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:366)
> ... 16 more
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 10 months
[JBoss JIRA] (TEIID-4626) Add a full snapshot refresh strategy based upon table modifications
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4626?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4626:
---------------------------------------
Observations:
Incoming objects can contain procedures and views as well. You would need to recursively navigate to discover all source tables - but you just want a unique set.
Since the triggers can now show up in exported ddl, we may want some way of excluding them.
The trigger name is not guaranteed to be unique - but it is highly likely given the naming convention.
Alter view would require the triggers to be dropped/readded, but we can also just document that as a limitation.
> Add a full snapshot refresh strategy based upon table modifications
> -------------------------------------------------------------------
>
> Key: TEIID-4626
> URL: https://issues.jboss.org/browse/TEIID-4626
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 9.3
>
>
> using the existing full materialization refresh logic we can trigger a refresh based upon a notion of how dirty a table is based upon a proportion of rows updated to the rows in the table. This would also require some change to the status table to capture the number of transitive row updates. This could then be hooked up to debezium as the event source.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 10 months
[JBoss JIRA] (TEIID-4788) DDL vdb is expensive to process
by Steven Hawkins (JIRA)
Steven Hawkins created TEIID-4788:
-------------------------------------
Summary: DDL vdb is expensive to process
Key: TEIID-4788
URL: https://issues.jboss.org/browse/TEIID-4788
Project: Teiid
Issue Type: Quality Risk
Components: Query Engine
Reporter: Steven Hawkins
Assignee: Steven Hawkins
Fix For: 9.3, 9.2.1
For many ddl operations, a fresh transformationmetadata instance is created, which requires converting the Database to runtime form. The larger the DDL vdb, the more expensive this becomes.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 10 months
[JBoss JIRA] (TEIID-4626) Add a full snapshot refresh strategy based upon table modifications
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4626?page=com.atlassian.jira.plugin... ]
Ramesh Reddy edited comment on TEIID-4626 at 2/24/17 2:04 PM:
--------------------------------------------------------------
That part I agree with :) In that case, I will introduce additional property like {{MATVIEW_ALLOWED_STALENESS_PCT}} and Default this to 1%, where we can calculate the current staleness using
{code}
stalecount/cardinality
{code}
One the threshold is reached, will flip the {{Status}} table Status to "NEEDS_LOADING" to let it load.
was (Author: rareddy):
That part I agree with :) In that case, I will introduce additional property like {{MATVIEW_ALLOWED_STALENESS_PCT}} and Default this to 1%, where we can calculate the current staleness using
{code}
cardinality/stalecount
{code}
One the threshold is reached, will flip the {{Status}} table Status to "NEEDS_LOADING" to let it load.
> Add a full snapshot refresh strategy based upon table modifications
> -------------------------------------------------------------------
>
> Key: TEIID-4626
> URL: https://issues.jboss.org/browse/TEIID-4626
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 9.3
>
>
> using the existing full materialization refresh logic we can trigger a refresh based upon a notion of how dirty a table is based upon a proportion of rows updated to the rows in the table. This would also require some change to the status table to capture the number of transitive row updates. This could then be hooked up to debezium as the event source.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 10 months
[JBoss JIRA] (TEIID-4626) Add a full snapshot refresh strategy based upon table modifications
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4626?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4626:
-------------------------------------
That part I agree with :) In that case, I will introduce additional property like {{MATVIEW_ALLOWED_STALENESS_PCT}} and Default this to 1%, where we can calculate the current staleness using
{code}
cardinality/stalecount
{code}
One the threshold is reached, will flip the {{Status}} table Status to "NEEDS_LOADING" to let it load.
> Add a full snapshot refresh strategy based upon table modifications
> -------------------------------------------------------------------
>
> Key: TEIID-4626
> URL: https://issues.jboss.org/browse/TEIID-4626
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 9.3
>
>
> using the existing full materialization refresh logic we can trigger a refresh based upon a notion of how dirty a table is based upon a proportion of rows updated to the rows in the table. This would also require some change to the status table to capture the number of transitive row updates. This could then be hooked up to debezium as the event source.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 10 months
[JBoss JIRA] (TEIID-4626) Add a full snapshot refresh strategy based upon table modifications
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4626?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4626:
---------------------------------------
The intent of this issue is to make sufficient approximations so that we can offer a somewhat lazy strategy beyond just TTL reload out-of-the-box. If there is much user complexity to this beyond a rough dirty estimate, then it's not worth pursuing.
> Add a full snapshot refresh strategy based upon table modifications
> -------------------------------------------------------------------
>
> Key: TEIID-4626
> URL: https://issues.jboss.org/browse/TEIID-4626
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 9.3
>
>
> using the existing full materialization refresh logic we can trigger a refresh based upon a notion of how dirty a table is based upon a proportion of rows updated to the rows in the table. This would also require some change to the status table to capture the number of transitive row updates. This could then be hooked up to debezium as the event source.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 10 months
[JBoss JIRA] (TEIID-4626) Add a full snapshot refresh strategy based upon table modifications
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4626?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4626:
-------------------------------------
Since we are just collecting # of source updates without correlation to view, it is unknown #rows or #columns on the view that are affected. Thus my comment about a possible callback procedure that user defined to make decision when to run the reload.
> Add a full snapshot refresh strategy based upon table modifications
> -------------------------------------------------------------------
>
> Key: TEIID-4626
> URL: https://issues.jboss.org/browse/TEIID-4626
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 9.3
>
>
> using the existing full materialization refresh logic we can trigger a refresh based upon a notion of how dirty a table is based upon a proportion of rows updated to the rows in the table. This would also require some change to the status table to capture the number of transitive row updates. This could then be hooked up to debezium as the event source.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years, 10 months