[JBoss JIRA] (TEIID-4441) Add FTP/SFTP based access to the File Resource Adapter
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4441?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4441:
-------------------------------------
One suggestion I have is do not download the file from FTP server and create a temp file locally, consider that as a stream of data. The metadata exposed from file translator already returns either Clob or Blob, so use those interfaces to pass in the contents of a file. Consider, when we do the same with HDFS this could be multi GB data at best or even larger, we should not to be making copies of these files.
> Add FTP/SFTP based access to the File Resource Adapter
> ------------------------------------------------------
>
> Key: TEIID-4441
> URL: https://issues.jboss.org/browse/TEIID-4441
> Project: Teiid
> Issue Type: Enhancement
> Components: Misc. Connectors
> Reporter: Ramesh Reddy
> Assignee: Kylin Soong
> Fix For: Open To Community, 9.2
>
>
> This has been requested few times in the community, it would be nice to give provide this functionality. Using http://commons.apache.org/proper/commons-net/ seems to be right decision.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4605) SqlServer translator - insert/update statements - timestamp is cast to datetime
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4605?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-4605.
-----------------------------------
Resolution: Done
Added the type mapping and updated associated literal and conversion functions.
> SqlServer translator - insert/update statements - timestamp is cast to datetime
> -------------------------------------------------------------------------------
>
> Key: TEIID-4605
> URL: https://issues.jboss.org/browse/TEIID-4605
> Project: Teiid
> Issue Type: Bug
> Components: JDBC Connector
> Affects Versions: 8.12.5
> Reporter: Juraj Duráni
> Assignee: Steven Hawkins
> Fix For: 9.2, 9.1.2
>
>
> For update statements (insert, update), *timestamp* values are cast to *datatime* type. In MSSQL, this type's range start from year 1753 (1753\-01\-01...). However, there is *datetime2* type, which start from 0001\-01\-01.
> This default cast to *datetime* prevents user from inserting value before year 1753 even if column in MSSQL is defined as datetime2.
> Note: Specifying column option NATIVE_TYPE has no effect on this issue.
> {code:plain|title=Stacktrace}
> Connector worker process failed for atomic-request=7owuBqBS/Vow.1.0.3: org.teiid.translator.jdbc.JDBCExecutionException: 242 TEIID11013:TEIID11004 Error executing statement(s): [SQL: UPDATE SmallA SET TimestampValue = CAST('0100-02-03 04:05:06.0' AS DATETIME)]
> at org.teiid.translator.jdbc.JDBCUpdateExecution.executeTranslatedCommand(JDBCUpdateExecution.java:293) [translator-jdbc-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.teiid.translator.jdbc.JDBCUpdateExecution.execute(JDBCUpdateExecution.java:84) [translator-jdbc-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem$1.execute(ConnectorWorkItem.java:402)
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:364)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0-internal]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)
> at com.sun.proxy.$Proxy48.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.8.0-internal]
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The conversion of a varchar data type to a datetime data type resulted in an out-of-range value.
> at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
> at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:254)
> at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:84)
> at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1510)
> at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:792)
> at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:689)
> at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
> at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
> at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
> at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
> at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeUpdate(SQLServerStatement.java:642)
> at org.jboss.jca.adapters.jdbc.WrappedStatement.executeUpdate(WrappedStatement.java:375)
> at org.teiid.translator.jdbc.JDBCUpdateExecution.executeTranslatedCommand(JDBCUpdateExecution.java:207) [translator-jdbc-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> ... 20 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4622) Adding new data quality functions from osDQ
by vivek singh (JIRA)
vivek singh created TEIID-4622:
----------------------------------
Summary: Adding new data quality functions from osDQ
Key: TEIID-4622
URL: https://issues.jboss.org/browse/TEIID-4622
Project: Teiid
Issue Type: Feature Request
Components: Common
Affects Versions: Open To Community
Environment: osdq-core and its dependency simmetric 3.0.0 is needed to run these functions
maven central location
http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22osdq-core%22
Reporter: vivek singh
Assignee: Steven Hawkins
Fix For: Open To Community
Attachments: DateUtil.java, NumberUtil.java, StringUtil.java
Following functions have been added into osdq-core for integration with teiid.
String functions :
// Encrypting a string using key and AES encryption
String encrypt ( String val, String key)
String decrypt ( String val, String key)
// Fuzzy distance - similarity between two strings
float levestheindistance ( String a, String b)
float jarowinklerdistance( String a , String b)
float cosinedistance (String a, String b)
// Format check for PII ( personal information)
boolean isCreditCardFormat(String a)
boolean isSSNFormat(String a)
boolean isPhoneFormat(String a)
boolean isEmailFormat(String a)
boolean hasWhiteSpace(String a )
Date Functions
// converting date into universal recognized format
long dateToEpoch(java.util.Date date)
Date epochToDate(long epoch)
Date epochToDate(long epoch, String TimeZone)
long diffInSec(date a, date b)
String convertToFormat ( Date a, String Format)
Numerical Functions
// Statistical functions
double stdDev(Number[] numSeries)
double range(Number[] numSeries)
double median(Number[] numSeries)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4611) Infinispan DSL translator - error deleting rows from table
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4611?page=com.atlassian.jira.plugin... ]
Work on TEIID-4611 started by Van Halbert.
------------------------------------------
> Infinispan DSL translator - error deleting rows from table
> ----------------------------------------------------------
>
> Key: TEIID-4611
> URL: https://issues.jboss.org/browse/TEIID-4611
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.8.6_3
> Reporter: Juraj Duráni
> Assignee: Van Halbert
> Priority: Blocker
>
> DSL translator is not able to delete rows from table:
> {code:sql|title=Example queries}
> insert into smalla (intkey, stringkey) values (1, '1');
> insert into smalla (intkey, stringkey) values (2, '2');
> delete from smalla
> {code}
> Teiid seems to try to delete record repeatedly:
> {code:plain|title=Exception}
> 14:05:20,351 WARN [org.teiid.CONNECTOR] (Worker1_QueryProcessorQueue37) TEIID21013 ObjectUpdateExecution: Warning - The object does not exist when trying to delete from table SmallA with key 2
> 14:05:20,352 WARN [org.teiid.CONNECTOR] (Worker1_QueryProcessorQueue37) Connector worker process failed for atomic-request=IuoK2RpItaN2.13.0.6: org.teiid.translator.TranslatorException: TEIID21012 Processing error: Unable to delete the object in table SmallA with key 2
> at org.teiid.translator.object.ObjectUpdateExecution.handleDelete(ObjectUpdateExecution.java:385) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.executeUpdate(ObjectUpdateExecution.java:137) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.execute(ObjectUpdateExecution.java:100) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem$1.execute(ConnectorWorkItem.java:404) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:366) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0-internal]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at com.sun.proxy.$Proxy47.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.8.0-internal]
> {code}
> Moreover, Teiid removes only one row (I will log this issue separately):
> *Data in table before delete:*
> |IntKey|StringKey|...|
> |1|1|...|
> |2|2|...|
> *Data in table after delete:*
> |IntKey|StringKey|...|
> |2|2|...|
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4620) MongoDB: Model "_id" during metadata import as string data type
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4620?page=com.atlassian.jira.plugin... ]
Ramesh Reddy resolved TEIID-4620.
---------------------------------
Resolution: Done
Labels: Alpha2 (was: )
Converted the "_id" type when importing an existing collection from MongoDB to a "string" runtime type in Teiid. Previously this was set as "object", however the value was being treated as hex string from before.
> MongoDB: Model "_id" during metadata import as string data type
> ---------------------------------------------------------------
>
> Key: TEIID-4620
> URL: https://issues.jboss.org/browse/TEIID-4620
> Project: Teiid
> Issue Type: Enhancement
> Components: Misc. Connectors
> Affects Versions: 8.7
> Reporter: Ramesh Reddy
> Assignee: Ramesh Reddy
> Labels: Alpha2
> Fix For: 9.2
>
>
> Currently when metadata is imported through NATIVE metadata processor, the "_id" column of the MongoDB collection is represented as "Object" but the value is always handled as "string". Convert this to a string type such that is column then can be used in criteria columns.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years