[JBoss JIRA] Resolved: (TEIID-24) Correlated reference in aggregate function of subquery not supported by all data sources
by Steven Hawkins (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-24?page=com.atlassian.jira.plugi... ]
Steven Hawkins resolved TEIID-24.
---------------------------------
Resolution: Out of Date
The rewriter will now catch situations where expressions that will become constants are used in aggregates. If a source has a problem with a correlated reference as part of a non-constant expression in an aggregate, then we'll open a new issue for that.
> Correlated reference in aggregate function of subquery not supported by all data sources
> ----------------------------------------------------------------------------------------
>
> Key: TEIID-24
> URL: https://jira.jboss.org/jira/browse/TEIID-24
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Connector
> Affects Versions: 6.x
> Reporter: Steven Hawkins
> Priority: Optional
> Fix For: 6.x
>
>
> Defect Tracker #10437: The below BQT query fails for Sybase or SQL Server because of the correlated reference inside the aggregate function, but works in Oracle or if processed by our own system. Seems like connector capabilities need to handle this but they aren't fine-grained enough currently.
> SELECT
> sa.intkey, sa.stringkey, sa.datevalue, sa.floatnum, sa.doublenum, sa.datevalue, sa.charvalue, sa.shortvalue, sa.objectvalue
> FROM
> bqt1.smalla AS sa
> WHERE
> sa.datevalue = (SELECT MAX(sb.datevalue) FROM bqt1.smalla AS sb WHERE (sb.intkey = sa.intkey) AND (sa.stringkey = sb.stringkey) )
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 8 months
[JBoss JIRA] Commented: (TEIID-244) Salesforce Connector should offer the abilty to query about object deletions.
by John Doyle (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-244?page=com.atlassian.jira.plug... ]
John Doyle commented on TEIID-244:
----------------------------------
Checked in the connector part of this implementation. Have to complete the importer generation of the physical procedures.
> Salesforce Connector should offer the abilty to query about object deletions.
> -----------------------------------------------------------------------------
>
> Key: TEIID-244
> URL: https://jira.jboss.org/jira/browse/TEIID-244
> Project: Teiid
> Issue Type: Feature Request
> Components: Salesforce Connector
> Affects Versions: 6.0.0
> Reporter: John Doyle
> Assignee: John Doyle
> Fix For: 6.1.0
>
>
> Genentech wants to materialize lots of SalesForce data in Oracle, to speed up their reporting query times. It makes a significant difference because SFDC does not support joins, whereas Oracle does. Wanting the best of both worlds, they also want the data to be as fresh as possible -- ideally, the mat views would be no more than 10 minutes out-of-sync with the live data.
> The amount of data in SalesForce already takes 20+ minutes to cache, and it will slowly grow over time. So, the current materialization process will not meet the cache coherence (freshness) requirement, since the data will be stale by the time the staging table is populated and swapped in.
> How would you speed it up?
> .......response....
> I actually put together a custom materialization script
> (attached,with doc and sample config file) for Credit Suisse for doing
> these sorts of partial refreshes. It is Oracle-specific, but since
> that is what you are doing too it should work for you too.
> However, the use case at Credit Suisse was for doing these partial
> refreshes nightly, with full refreshes done weekly. There is (just as
> with our standard materialization scripts) a short period (when the
> table names are being swapped) when the materialization won't be stable
> (queries could fail or return unexpected results). I don't know if
> that makes it not suitable for doing 10 minute refreshes.
> I would recommend instead using the materialization only for historical
> data (from overnight materialization run), and unioning it to the live
> data for the newer stuff. That is, query SFDC only for data where
> createdDate or LastModifiedDate = today, and union that with the
> current data. Two issues you'd need to deal with in your logic:
> -if the same record is retrieved from both sources (meaning it was an
> existing record that had been modified today), the historical record
> should be discarded in favor of the live one.
> -deleted records - how to detect records that were deleted today? If
> you could ask SFDC exactly what was deleted today that would be great,
> but that is probably wishful thinking. So failing that you would need
> to ask SFDC for the keys for all live records, and then discard from
> the historical data any records with keys not on that list. Or, you
> could just live with having the deleted records in the view for an
> extra day...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 8 months
[JBoss JIRA] Work started: (TEIID-244) Salesforce Connector should offer the abilty to query about object deletions.
by John Doyle (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-244?page=com.atlassian.jira.plug... ]
Work on TEIID-244 started by John Doyle.
> Salesforce Connector should offer the abilty to query about object deletions.
> -----------------------------------------------------------------------------
>
> Key: TEIID-244
> URL: https://jira.jboss.org/jira/browse/TEIID-244
> Project: Teiid
> Issue Type: Feature Request
> Components: Salesforce Connector
> Affects Versions: 6.0.0
> Reporter: John Doyle
> Assignee: John Doyle
> Fix For: 6.1.0
>
>
> Genentech wants to materialize lots of SalesForce data in Oracle, to speed up their reporting query times. It makes a significant difference because SFDC does not support joins, whereas Oracle does. Wanting the best of both worlds, they also want the data to be as fresh as possible -- ideally, the mat views would be no more than 10 minutes out-of-sync with the live data.
> The amount of data in SalesForce already takes 20+ minutes to cache, and it will slowly grow over time. So, the current materialization process will not meet the cache coherence (freshness) requirement, since the data will be stale by the time the staging table is populated and swapped in.
> How would you speed it up?
> .......response....
> I actually put together a custom materialization script
> (attached,with doc and sample config file) for Credit Suisse for doing
> these sorts of partial refreshes. It is Oracle-specific, but since
> that is what you are doing too it should work for you too.
> However, the use case at Credit Suisse was for doing these partial
> refreshes nightly, with full refreshes done weekly. There is (just as
> with our standard materialization scripts) a short period (when the
> table names are being swapped) when the materialization won't be stable
> (queries could fail or return unexpected results). I don't know if
> that makes it not suitable for doing 10 minute refreshes.
> I would recommend instead using the materialization only for historical
> data (from overnight materialization run), and unioning it to the live
> data for the newer stuff. That is, query SFDC only for data where
> createdDate or LastModifiedDate = today, and union that with the
> current data. Two issues you'd need to deal with in your logic:
> -if the same record is retrieved from both sources (meaning it was an
> existing record that had been modified today), the historical record
> should be discarded in favor of the live one.
> -deleted records - how to detect records that were deleted today? If
> you could ask SFDC exactly what was deleted today that would be great,
> but that is probably wishful thinking. So failing that you would need
> to ask SFDC for the keys for all live records, and then discard from
> the historical data any records with keys not on that list. Or, you
> could just live with having the deleted records in the view for an
> extra day...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 8 months
[JBoss JIRA] Commented: (TEIID-233) Server should support JGroups communication over P2P
by Ramesh Reddy (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-233?page=com.atlassian.jira.plug... ]
Ramesh Reddy commented on TEIID-233:
------------------------------------
There is discussion thread on this. The summary is of it is
The user would have those three deployment options. Our default would be Unicast over TCP using an embedded router that supports fail-over. Option two would be Multicast over UDP (what we do now by default) and option three would be Unicast over UDP or TCP using GossipRouter.
Further discussions as to implementations of Option 1, the following is being considered
When a primary server process starts, it will start a gossip server. When a secondary host being added, the installation requires the user to specify the primary server, at the same this can be extended such that user has to specify the gossip port of the primary server, and this information is saved along with bootstrap properties file. The primary server's bootstrap properties either now needs to be updated manually or automatically about the second server, so that when either server goes down and comes back up, they know about each other to cluster back up.
> Server should support JGroups communication over P2P
> ----------------------------------------------------
>
> Key: TEIID-233
> URL: https://jira.jboss.org/jira/browse/TEIID-233
> Project: Teiid
> Issue Type: Feature Request
> Components: Server
> Affects Versions: 6.0.0
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Fix For: 6.1.0
>
>
> Currently Server only supports the JGroups over Multicast we need to support unicast protocol tool, with configuration.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 8 months
[JBoss JIRA] Assigned: (TEIID-521) Prepared statement methods addBatch() and clearBatch() can cause NullPointerException
by Ramesh Reddy (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-521?page=com.atlassian.jira.plug... ]
Ramesh Reddy reassigned TEIID-521:
----------------------------------
Assignee: Larry O'Leary
> Prepared statement methods addBatch() and clearBatch() can cause NullPointerException
> -------------------------------------------------------------------------------------
>
> Key: TEIID-521
> URL: https://jira.jboss.org/jira/browse/TEIID-521
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Connector
> Affects Versions: 6.0.0
> Reporter: Larry O'Leary
> Assignee: Larry O'Leary
> Fix For: 6.1.0
>
> Attachments: TestMMPreparedStatement.java
>
>
> When attempting to debug an unrelated issue I discovered that the batch processing methods on a prepared statement can result in a NullPointerException depending on when the methods are called.
> For example:
> java.lang.NullPointerException
> at com.metamatrix.jdbc.MMStatement.clearBatch(MMStatement.java:274)
> at com.metamatrix.jdbc.TestPreparedStatementBatchedUpdate.testInserts(TestPreparedStatementBatchedUpdate.java:102)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:585)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
> at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Is a result of calling clearBatch() on a statement after it was done executing a batched update. It appears this is simply a need of NOT calling clearBatch() at the wrong time but I feel that we should gracefully handle this situation as a clearBatch() call may appear in a try/catch/finally block.
> The same goes for addBatch() if no parameters have been set. Not sure if we should actually throw an exception in such a case or not as it makes no sense to call addBatch() with no parameters defined but it would also make sense that we would not throw a nasty NPE in such a case.
> I have only seen this on R050502 and R050503 and have not looked at R050504c or Teiid.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 8 months
[JBoss JIRA] Updated: (TEIID-521) Prepared statement methods addBatch() and clearBatch() can cause NullPointerException
by Larry O'Leary (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-521?page=com.atlassian.jira.plug... ]
Larry O'Leary updated TEIID-521:
--------------------------------
Attachment: TestMMPreparedStatement.java
Here are some basic UnitTests that cover this issue. It appears that addBatch() does not call the NPE in Teiid as it did in 5.5.x but the UnitTest still fails for other reasons. If you want me to fix, feel free to reassign to me.
> Prepared statement methods addBatch() and clearBatch() can cause NullPointerException
> -------------------------------------------------------------------------------------
>
> Key: TEIID-521
> URL: https://jira.jboss.org/jira/browse/TEIID-521
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Connector
> Affects Versions: 6.0.0
> Reporter: Larry O'Leary
> Fix For: 6.1.0
>
> Attachments: TestMMPreparedStatement.java
>
>
> When attempting to debug an unrelated issue I discovered that the batch processing methods on a prepared statement can result in a NullPointerException depending on when the methods are called.
> For example:
> java.lang.NullPointerException
> at com.metamatrix.jdbc.MMStatement.clearBatch(MMStatement.java:274)
> at com.metamatrix.jdbc.TestPreparedStatementBatchedUpdate.testInserts(TestPreparedStatementBatchedUpdate.java:102)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:585)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
> at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Is a result of calling clearBatch() on a statement after it was done executing a batched update. It appears this is simply a need of NOT calling clearBatch() at the wrong time but I feel that we should gracefully handle this situation as a clearBatch() call may appear in a try/catch/finally block.
> The same goes for addBatch() if no parameters have been set. Not sure if we should actually throw an exception in such a case or not as it makes no sense to call addBatch() with no parameters defined but it would also make sense that we would not throw a nasty NPE in such a case.
> I have only seen this on R050502 and R050503 and have not looked at R050504c or Teiid.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 8 months
[JBoss JIRA] Created: (TEIID-531) RelationalPlan.clone() causes NullPointerException when cloning BatchedUpdatePlan
by Larry O'Leary (JIRA)
RelationalPlan.clone() causes NullPointerException when cloning BatchedUpdatePlan
---------------------------------------------------------------------------------
Key: TEIID-531
URL: https://jira.jboss.org/jira/browse/TEIID-531
Project: Teiid
Issue Type: Bug
Components: Query Engine
Affects Versions: 6.0.0, 6.1.0
Reporter: Larry O'Leary
Assignee: Larry O'Leary
Priority: Minor
When executing a new unit test which used a BatchedUpdateCommand with a PreparedStatement the following error would occur when executing ProcessorPlan.clone() using a plan of type BatchedUpdatePlan:
BatchedUpdatePlan {
ProjectNode(2) output=["Count"] ["Count"]
BatchedUpdateNode(3) output=["Count"]
}
java.lang.RuntimeException: java.lang.NullPointerException
at com.metamatrix.query.processor.TestProcessor.helpProcess(TestProcessor.java:224)
at org.teiid.dqp.internal.process.TestPreparedStatement.helpTestProcessing(TestPreparedStatement.java:127)
at org.teiid.dqp.internal.process.TestPreparedStatementBatchedUpdate.testProcessor_VarNumCmds(TestPreparedStatementBatchedUpdate.java:196)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:81)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:45)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
Caused by: java.lang.NullPointerException
at java.util.ArrayList.<init>(ArrayList.java:131)
at com.metamatrix.query.processor.relational.RelationalPlan.clone(RelationalPlan.java:136)
at com.metamatrix.query.processor.batch.BatchedUpdatePlan.clone(BatchedUpdatePlan.java:83)
at com.metamatrix.query.processor.TestProcessor.helpProcess(TestProcessor.java:214)
... 21 more
It appears that in the test case outputCols is null in the ProjectNode of the relational plan which then results in the clone of outputcols to fail due to it being used as a parameter to the creation of the new ArrayList.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 8 months
[JBoss JIRA] Commented: (TEIID-521) Prepared statement methods addBatch() and clearBatch() can cause NullPointerException
by Ramesh Reddy (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-521?page=com.atlassian.jira.plug... ]
Ramesh Reddy commented on TEIID-521:
------------------------------------
Can you attach the test case you may be working with.
> Prepared statement methods addBatch() and clearBatch() can cause NullPointerException
> -------------------------------------------------------------------------------------
>
> Key: TEIID-521
> URL: https://jira.jboss.org/jira/browse/TEIID-521
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Connector
> Affects Versions: 6.0.0
> Reporter: Larry O'Leary
> Fix For: 6.1.0
>
>
> When attempting to debug an unrelated issue I discovered that the batch processing methods on a prepared statement can result in a NullPointerException depending on when the methods are called.
> For example:
> java.lang.NullPointerException
> at com.metamatrix.jdbc.MMStatement.clearBatch(MMStatement.java:274)
> at com.metamatrix.jdbc.TestPreparedStatementBatchedUpdate.testInserts(TestPreparedStatementBatchedUpdate.java:102)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:585)
> at junit.framework.TestCase.runTest(TestCase.java:154)
> at junit.framework.TestCase.runBare(TestCase.java:127)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:118)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at junit.framework.TestSuite.runTest(TestSuite.java:208)
> at junit.framework.TestSuite.run(TestSuite.java:203)
> at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
> at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Is a result of calling clearBatch() on a statement after it was done executing a batched update. It appears this is simply a need of NOT calling clearBatch() at the wrong time but I feel that we should gracefully handle this situation as a clearBatch() call may appear in a try/catch/finally block.
> The same goes for addBatch() if no parameters have been set. Not sure if we should actually throw an exception in such a case or not as it makes no sense to call addBatch() with no parameters defined but it would also make sense that we would not throw a nasty NPE in such a case.
> I have only seen this on R050502 and R050503 and have not looked at R050504c or Teiid.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 8 months