[JBoss JIRA] (TEIID-4992) Add OSDQ set functions
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4992?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-4992:
---------------------------------------
I think we need a formal definition of what is expected.
Using the example from before:
String[] first = new String[]{"vivek16","abcd","vivek44", "x", "xx"};
String[] second = new String[]{"vivek36","vivekkk","abcd","vivek445", "x"};
If we do something like:
{code}
for (float f : Arrays.asList(0.0f, 1f, 1.1f)) {
System.out.println(Arrays.toString(intersectionFuzzy(first,second,f)));
System.out.println(Arrays.toString(intersectionFuzzy(second,first,f)));
System.out.println(Arrays.toString(minusFuzzy(first,second,f)));
System.out.println(Arrays.toString(minusFuzzy(second,first,f)));
System.out.println(Arrays.toString(unionFuzzy(first,second,f)));
System.out.println(Arrays.toString(unionFuzzy(second,first,f)));
}
{code}
We see the output:
{code}
[abcd, x, xx]
[abcd, vivekkk, x]
[]
[]
[abcd]
[abcd]
[abcd, xx]
[abcd, x]
[vivek16, vivek44]
[vivek36, vivek445, vivekkk]
[abcd, vivek16, vivek36, vivek44, vivek445, vivekkk, x]
[abcd, vivek16, vivek36, vivek44, vivek445, vivekkk, x]
[abcd, x]
[abcd, x]
[vivek16, vivek44, xx]
[vivek36, vivek445, vivekkk]
[abcd, vivek16, vivek36, vivek44, vivek445, vivekkk, x, xx]
[abcd, vivek16, vivek36, vivek44, vivek445, vivekkk, x, xx]
{code}
Some things that may need clarifying are:
intersectionFuzzy(first,second,0.0f) - does not contain any vivek entry, but it does when the arguments are reversed.
intersectionFuzzy(first,second,0.0f) - with the arguments in either order contain more results than the similar unionFuzzy
intersectionFuzzy(first,second,1.0f) - has asymmetric results depending on the order of the inputs
> Add OSDQ set functions
> ----------------------
>
> Key: TEIID-4992
> URL: https://issues.jboss.org/browse/TEIID-4992
> Project: Teiid
> Issue Type: Enhancement
> Components: Query Engine
> Reporter: Steven Hawkins
> Assignee: Steven Hawkins
> Fix For: 10.0
>
> Attachments: SetUtil.java
>
>
> There will be a new osdq core library drop in maven central soon containing additional functions for Teiid to add.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 3 months
[JBoss JIRA] (TEIID-5101) .vdb with JDG data source, with NATIVE defined, throws exception
by Van Halbert (JIRA)
Van Halbert created TEIID-5101:
----------------------------------
Summary: .vdb with JDG data source, with NATIVE defined, throws exception
Key: TEIID-5101
URL: https://issues.jboss.org/browse/TEIID-5101
Project: Teiid
Issue Type: Bug
Components: Infinispan
Affects Versions: 8.12.x-6.4
Reporter: Van Halbert
Assignee: Ramesh Reddy
Priority: Blocker
When a .vdb (created in designer), and the internal vdb.xml contains the <metadata type="NATIVE"></metadata>, the vdb will not deploy.
However, it will work if the .vdb is exported to vdb xml, because the DDL and the NATIVE tags are there.
Guessing that the translator doesn't know how to handle a .vdb file.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 3 months
[JBoss JIRA] (TEIID-5092) Joining tables from S3 occasionaly fails with NumberFormatException
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5092?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-5092.
-----------------------------------
Fix Version/s: 10.0
8.12.x-6.4
Resolution: Done
Inlined the usage of the date format.
> Joining tables from S3 occasionaly fails with NumberFormatException
> -------------------------------------------------------------------
>
> Key: TEIID-5092
> URL: https://issues.jboss.org/browse/TEIID-5092
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.12.x-6.4
> Reporter: Andrej Šmigala
> Assignee: Steven Hawkins
> Priority: Critical
> Fix For: 10.0, 8.12.x-6.4
>
>
> Running any query that reads multiple tables from an Amazon S3 datasource sometimes fails with the following exception:
> {noformat}
> [org.teiid.CONNECTOR] (Worker0_QueryProcessorQueue3373) Connector worker process failed for atomic-request=ZwjBqkPjJdw2.386.4.689: java.lang.NumberFormatException: For input string: ""
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) [rt.jar:1.8.0_131]
> at java.lang.Long.parseLong(Long.java:601) [rt.jar:1.8.0_131]
> at java.lang.Long.parseLong(Long.java:631) [rt.jar:1.8.0_131]
> at java.text.DigitList.getLong(DigitList.java:195) [rt.jar:1.8.0_131]
> at java.text.DecimalFormat.parse(DecimalFormat.java:2051) [rt.jar:1.8.0_131]
> at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:1869) [rt.jar:1.8.0_131]
> at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1514) [rt.jar:1.8.0_131]
> at java.text.DateFormat.parse(DateFormat.java:364) [rt.jar:1.8.0_131]
> at org.teiid.translator.amazon.s3.S3ProcedureExecution.next(S3ProcedureExecution.java:488) [translator-amazon-s3-8.12.11.6_4-redhat-64-6.jar:8.12.11.6_4-redhat-64-6]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.handleBatch(ConnectorWorkItem.java:433)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.more(ConnectorWorkItem.java:236)
> at sun.reflect.GeneratedMethodAccessor211.invoke(Unknown Source) [:1.8.0_131]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_131]
> at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_131]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy79.more(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:309)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_131]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_131]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_131]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_131]
> {noformat}.
> The actual input string reffered to in the stack trace occasionaly contains garbage data.
> This is caused by unsynchronized access to a static SimpleDateFormat field here: https://github.com/teiid/teiid/blob/64-8.12.x/connectors/translator-amazo...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 3 months
[JBoss JIRA] (TEIID-5097) Cannot run time-based queries against Osisoft PI
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5097?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-5097:
---------------------------------------
Changing to a major quality risk, not a blocker. This is the proper functioning of our type / comparison logic. We do not generally support custom literal conversions of the source.
In later versions by default comparisons such as these will require an explicit cast to clarify that a conversion is occurring.
There are standard interval types, but the literals above appear to be a custom specific to pi. Is there a reference as to what is desired to be supported?
> Cannot run time-based queries against Osisoft PI
> ------------------------------------------------
>
> Key: TEIID-5097
> URL: https://issues.jboss.org/browse/TEIID-5097
> Project: Teiid
> Issue Type: Quality Risk
> Components: JDBC Connector
> Affects Versions: 8.12.x-6.4
> Reporter: Andrej Šmigala
> Assignee: Steven Hawkins
>
> Osisoft PI supports a relative time literals syntax, e.g.
> {code:sql}
> select * from dvqe.Data.Archive a where a.time between '*-14d' and '*'
> {code}
> will select all data between right now and 14 days ago, and
> {code:sql}
> select * from dvqe.Data.Archive a where a.time > 'y'
> {code}
> will select all data after yesterday midnight.
> The string literals are converted to time values in the PI Server
> Running the same queries through teiid however returns incorrect results, because teiid pushes a cast to string on the Time column, which results in string comparison on the datasource:
> {code:sql|title=Pushed query}
> SELECT TOP 100 cast(g_0.[ElementAttributeID] as String), g_0.[Time] AS c_1, g_0.[Value] AS c_2, g_0.[ValueInt] AS c_3, g_0.
> [ValueDbl] AS c_4, g_0.[ValueStr] AS c_5, cast(g_0.[ValueGuid] as String), g_0.[ValueDateTime] AS c_7, g_0.[Status] AS c_8, g_0.[Annotated] AS c_9, g_0.[IsGood] A
> S c_10, g_0.[Questionable] AS c_11, g_0.[Substituted] AS c_12 FROM [dvqe].[Data].[Archive] AS g_0 WHERE cast(g_0.[Time] AS String) > 'y'
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 3 months
[JBoss JIRA] (TEIID-5097) Cannot run time-based queries against Osisoft PI
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-5097?page=com.atlassian.jira.plugin... ]
Steven Hawkins updated TEIID-5097:
----------------------------------
Issue Type: Quality Risk (was: Bug)
Priority: Major (was: Blocker)
> Cannot run time-based queries against Osisoft PI
> ------------------------------------------------
>
> Key: TEIID-5097
> URL: https://issues.jboss.org/browse/TEIID-5097
> Project: Teiid
> Issue Type: Quality Risk
> Components: JDBC Connector
> Affects Versions: 8.12.x-6.4
> Reporter: Andrej Šmigala
> Assignee: Steven Hawkins
>
> Osisoft PI supports a relative time literals syntax, e.g.
> {code:sql}
> select * from dvqe.Data.Archive a where a.time between '*-14d' and '*'
> {code}
> will select all data between right now and 14 days ago, and
> {code:sql}
> select * from dvqe.Data.Archive a where a.time > 'y'
> {code}
> will select all data after yesterday midnight.
> The string literals are converted to time values in the PI Server
> Running the same queries through teiid however returns incorrect results, because teiid pushes a cast to string on the Time column, which results in string comparison on the datasource:
> {code:sql|title=Pushed query}
> SELECT TOP 100 cast(g_0.[ElementAttributeID] as String), g_0.[Time] AS c_1, g_0.[Value] AS c_2, g_0.[ValueInt] AS c_3, g_0.
> [ValueDbl] AS c_4, g_0.[ValueStr] AS c_5, cast(g_0.[ValueGuid] as String), g_0.[ValueDateTime] AS c_7, g_0.[Status] AS c_8, g_0.[Annotated] AS c_9, g_0.[IsGood] A
> S c_10, g_0.[Questionable] AS c_11, g_0.[Substituted] AS c_12 FROM [dvqe].[Data].[Archive] AS g_0 WHERE cast(g_0.[Time] AS String) > 'y'
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 3 months