[JBoss JIRA] (TEIID-4611) Infinispan DSL translator - error deleting rows from table
by Van Halbert (JIRA)
[ https://issues.jboss.org/browse/TEIID-4611?page=com.atlassian.jira.plugin... ]
Work on TEIID-4611 stopped by Van Halbert.
------------------------------------------
> Infinispan DSL translator - error deleting rows from table
> ----------------------------------------------------------
>
> Key: TEIID-4611
> URL: https://issues.jboss.org/browse/TEIID-4611
> Project: Teiid
> Issue Type: Bug
> Components: JDG Connector, Misc. Connectors
> Affects Versions: 8.12.8.6_3
> Reporter: Juraj Duráni
> Assignee: Van Halbert
> Priority: Blocker
>
> DSL translator is not able to delete rows from table:
> {code:sql|title=Example queries}
> insert into smalla (intkey, stringkey) values (1, '1');
> insert into smalla (intkey, stringkey) values (2, '2');
> delete from smalla
> {code}
> Teiid seems to try to delete record repeatedly:
> {code:plain|title=Exception}
> 14:05:20,351 WARN [org.teiid.CONNECTOR] (Worker1_QueryProcessorQueue37) TEIID21013 ObjectUpdateExecution: Warning - The object does not exist when trying to delete from table SmallA with key 2
> 14:05:20,352 WARN [org.teiid.CONNECTOR] (Worker1_QueryProcessorQueue37) Connector worker process failed for atomic-request=IuoK2RpItaN2.13.0.6: org.teiid.translator.TranslatorException: TEIID21012 Processing error: Unable to delete the object in table SmallA with key 2
> at org.teiid.translator.object.ObjectUpdateExecution.handleDelete(ObjectUpdateExecution.java:385) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.executeUpdate(ObjectUpdateExecution.java:137) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.translator.object.ObjectUpdateExecution.execute(ObjectUpdateExecution.java:100) [translator-object-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem$1.execute(ConnectorWorkItem.java:404) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:366) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0-internal]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0-internal]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0-internal]
> at java.lang.reflect.Method.invoke(Method.java:483) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at com.sun.proxy.$Proxy47.execute(Unknown Source)
> at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:306) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0-internal]
> at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210) [teiid-engine-8.12.8.6_3-redhat-1.jar:8.12.8.6_3-redhat-1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0-internal]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0-internal]
> at java.lang.Thread.run(Thread.java:744) [rt.jar:1.8.0-internal]
> {code}
> Moreover, Teiid removes only one row (I will log this issue separately):
> *Data in table before delete:*
> |IntKey|StringKey|...|
> |1|1|...|
> |2|2|...|
> *Data in table after delete:*
> |IntKey|StringKey|...|
> |2|2|...|
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4584) SimpleDB doesn' work
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4584?page=com.atlassian.jira.plugin... ]
Ramesh Reddy reassigned TEIID-4584:
-----------------------------------
Assignee: Ramesh Reddy
> SimpleDB doesn' work
> --------------------
>
> Key: TEIID-4584
> URL: https://issues.jboss.org/browse/TEIID-4584
> Project: Teiid
> Issue Type: Bug
> Components: Connector API
> Affects Versions: 8.12.6.6_3
> Environment: RHEL 7.x, JBoss 6.4, JDV 6.3,
> JBDS 9.1.0.GA, GA-v20160414-0124-B497
> Also Fedora 23
> Reporter: Glen Wilcox
> Assignee: Ramesh Reddy
> Fix For: 9.2, 9.1.2
>
> Attachments: SimpleDB_ERROR_HTTP.txt, SimpleDB_ERROR_Jackson.txt
>
>
> I was trying to set up the simpledb example:
> https://developer.jboss.org/wiki/IntegrateAmazonSimpleDBWithTeiid
> When I create the DataSource to AWS I get the following error:
> Inline image 1
> and the stack trace:
> 07:19:29,048 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (teiid-async-threads - 1) IJ000604: Throwable while attempting to get a new connection: null: java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException
> at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:46)
> at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:41)
> at org.teiid.resource.spi.BasicManagedConnectionFactory.createManagedConnection(BasicManagedConnectionFactory.java:71) [teiid-api-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:834)
> at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:379)
> at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:453)
> at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:425)
> at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:354)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4584) SimpleDB doesn' work
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4584?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4584:
-------------------------------------
response from AWS
Hi @rareddy
SDK is modularized into multiple maven sub modules. SDK now have one maven sub module per AWS service. The aws-java-sdk module transitively depends on all sub modules. This is just a convenient modules for customers who wish to depend on entire SDK.
We modularized the SDK in 1.9 major version. Prior to 1.9 SDK was vended as one single maven module.
https://aws.amazon.com/blogs/developer/aws-sdk-for-java-maven-modules/
> SimpleDB doesn' work
> --------------------
>
> Key: TEIID-4584
> URL: https://issues.jboss.org/browse/TEIID-4584
> Project: Teiid
> Issue Type: Bug
> Components: Connector API
> Affects Versions: 8.12.6.6_3
> Environment: RHEL 7.x, JBoss 6.4, JDV 6.3,
> JBDS 9.1.0.GA, GA-v20160414-0124-B497
> Also Fedora 23
> Reporter: Glen Wilcox
> Fix For: 9.2, 9.1.2
>
> Attachments: SimpleDB_ERROR_HTTP.txt, SimpleDB_ERROR_Jackson.txt
>
>
> I was trying to set up the simpledb example:
> https://developer.jboss.org/wiki/IntegrateAmazonSimpleDBWithTeiid
> When I create the DataSource to AWS I get the following error:
> Inline image 1
> and the stack trace:
> 07:19:29,048 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (teiid-async-threads - 1) IJ000604: Throwable while attempting to get a new connection: null: java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException
> at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:46)
> at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:41)
> at org.teiid.resource.spi.BasicManagedConnectionFactory.createManagedConnection(BasicManagedConnectionFactory.java:71) [teiid-api-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:834)
> at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:379)
> at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:453)
> at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:425)
> at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:354)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4584) SimpleDB doesn' work
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4584?page=com.atlassian.jira.plugin... ]
Steven Hawkins reassigned TEIID-4584:
-------------------------------------
Description:
I was trying to set up the simpledb example:
https://developer.jboss.org/wiki/IntegrateAmazonSimpleDBWithTeiid
When I create the DataSource to AWS I get the following error:
Inline image 1
and the stack trace:
07:19:29,048 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (teiid-async-threads - 1) IJ000604: Throwable while attempting to get a new connection: null: java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException
at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:46)
at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:41)
at org.teiid.resource.spi.BasicManagedConnectionFactory.createManagedConnection(BasicManagedConnectionFactory.java:71) [teiid-api-8.12.5.redhat-8.jar:8.12.5.redhat-8]
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:834)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:379)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:453)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:425)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:354)
was:
I was trying to set up the simpledb example:
https://developer.jboss.org/wiki/IntegrateAmazonSimpleDBWithTeiid
When I create the DataSource to AWS I get the following error:
Inline image 1
and the stack trace:
07:19:29,048 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (teiid-async-threads - 1) IJ000604: Throwable while attempting to get a new connection: null: java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException
at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:46)
at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:41)
at org.teiid.resource.spi.BasicManagedConnectionFactory.createManagedConnection(BasicManagedConnectionFactory.java:71) [teiid-api-8.12.5.redhat-8.jar:8.12.5.redhat-8]
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:834)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:379)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:453)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:425)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:354)
Fix Version/s: 9.2
9.1.2
Assignee: (was: Steven Hawkins)
Looks like we'll need to redo the dependencies.
> SimpleDB doesn' work
> --------------------
>
> Key: TEIID-4584
> URL: https://issues.jboss.org/browse/TEIID-4584
> Project: Teiid
> Issue Type: Bug
> Components: Connector API
> Affects Versions: 8.12.6.6_3
> Environment: RHEL 7.x, JBoss 6.4, JDV 6.3,
> JBDS 9.1.0.GA, GA-v20160414-0124-B497
> Also Fedora 23
> Reporter: Glen Wilcox
> Fix For: 9.2, 9.1.2
>
> Attachments: SimpleDB_ERROR_HTTP.txt, SimpleDB_ERROR_Jackson.txt
>
>
> I was trying to set up the simpledb example:
> https://developer.jboss.org/wiki/IntegrateAmazonSimpleDBWithTeiid
> When I create the DataSource to AWS I get the following error:
> Inline image 1
> and the stack trace:
> 07:19:29,048 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (teiid-async-threads - 1) IJ000604: Throwable while attempting to get a new connection: null: java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException
> at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:46)
> at org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory$1.getConnection(SimpleDBManagedConnectionFactory.java:41)
> at org.teiid.resource.spi.BasicManagedConnectionFactory.createManagedConnection(BasicManagedConnectionFactory.java:71) [teiid-api-8.12.5.redhat-8.jar:8.12.5.redhat-8]
> at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:834)
> at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:379)
> at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:453)
> at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:425)
> at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:354)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4622) Adding new data quality functions from osDQ
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4622?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-4622:
-------------------------------------
Add these functions to "data-quality" project's function library. Make sure that they are not already provided through the standard Teiid function library.
> Adding new data quality functions from osDQ
> -------------------------------------------
>
> Key: TEIID-4622
> URL: https://issues.jboss.org/browse/TEIID-4622
> Project: Teiid
> Issue Type: Feature Request
> Components: Common
> Affects Versions: Open To Community
> Environment: osdq-core and its dependency simmetric 3.0.0 is needed to run these functions
> maven central location
> http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22osdq-core%22
> Reporter: vivek singh
> Assignee: Kylin Soong
> Fix For: Open To Community
>
> Attachments: DateUtil.java, NumberUtil.java, StringUtil.java
>
>
> Following functions have been added into osdq-core for integration with teiid.
> String functions :
> // Encrypting a string using key and AES encryption
> String encrypt ( String val, String key)
> String decrypt ( String val, String key)
> // Fuzzy distance - similarity between two strings
> float levestheindistance ( String a, String b)
> float jarowinklerdistance( String a , String b)
> float cosinedistance (String a, String b)
> // Format check for PII ( personal information)
> boolean isCreditCardFormat(String a)
> boolean isSSNFormat(String a)
> boolean isPhoneFormat(String a)
> boolean isEmailFormat(String a)
> boolean hasWhiteSpace(String a )
> Date Functions
> // converting date into universal recognized format
> long dateToEpoch(java.util.Date date)
> Date epochToDate(long epoch)
> Date epochToDate(long epoch, String TimeZone)
> long diffInSec(date a, date b)
> String convertToFormat ( Date a, String Format)
> Numerical Functions
> // Statistical functions
> double stdDev(Number[] numSeries)
> double range(Number[] numSeries)
> double median(Number[] numSeries)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4619) left join returns wrong results
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-4619?page=com.atlassian.jira.plugin... ]
Work on TEIID-4619 started by Steven Hawkins.
---------------------------------------------
> left join returns wrong results
> -------------------------------
>
> Key: TEIID-4619
> URL: https://issues.jboss.org/browse/TEIID-4619
> Project: Teiid
> Issue Type: Bug
> Affects Versions: 9.0.4
> Reporter: Bram Gadeyne
> Assignee: Steven Hawkins
> Priority: Critical
> Attachments: correct_result.txt, enclosed_queryplan.txt, wrong_result.txt
>
>
> I have the following situation.
> I have a temporary table #tmp_admissions that contains 8047 rows.
> In this first query there are 66290 results. However if I only look at the lines for infectionid 880 then there are only 16 lines.
> {code:sql}
> select l.infectionid, l.id as linkid, lc.linkcultureid, lc.responsibleculture, lc.culturealternative, cl.sampleinsertts, cl.specimennumber,
> cl.culturenumber, cl.culturename, cl.quotation, ls.material, ls.sampletime,
> abr.culturenumber as abgram_culturenumber,abr.antibiogrampart, abr.resisttype,
> lc.antibiogramculturenr,lc.antibiogramspecimennr,lc.antibiogramsampleinsertts
> from #tmp_admissions adm
> join cos2_links l on l.admissionid = cast(adm.patientid as string)
> join cos2_link_culture lc on lc.linkid = l.id
> left join cos2_lab_culture cl on cl.culturenumber = lc.culturenr and cl.specimennumber = lc.culturespecimennr and cl.sampleinsertts = lc.culturesampleinsertts
> left join cos2_lab_sample ls on ls.inserttime = cl.sampleinsertts and ls.specimennumber = cl.specimennumber
> left join cos2_antibiogramresistences abr on abr.specimennumber = cl.specimennumber and abr.culturenumber = cl.culturenumber and abr.sampleinsertts = cl.sampleinsertts
> {code}
> This query does almost the same but returns 30 rows (and is correct).
> {code:sql}
> select l.infectionid, l.id as linkid, lc.linkcultureid, lc.responsibleculture, lc.culturealternative, cl.sampleinsertts, cl.specimennumber,
> cl.culturenumber, cl.culturename, cl.quotation, ls.material, ls.sampletime,
> abr.culturenumber as abgram_culturenumber,abr.antibiogrampart, abr.resisttype,
> lc.antibiogramculturenr,lc.antibiogramspecimennr,lc.antibiogramsampleinsertts
> from cos2_links l
> join cos2_link_culture lc on lc.linkid = l.id
> left join cos2_lab_culture cl on cl.culturenumber = lc.culturenr and cl.specimennumber = lc.culturespecimennr and cl.sampleinsertts = lc.culturesampleinsertts
> left join cos2_lab_sample ls on ls.inserttime = cl.sampleinsertts and ls.specimennumber = cl.specimennumber
> left join cos2_antibiogramresistences abr on abr.specimennumber = cl.specimennumber and abr.culturenumber = cl.culturenumber and abr.sampleinsertts = cl.sampleinsertts
> where l.infectionid = 880
> {code}
> cos2_link_culture contains 2 rows for this infectionid. The left join statements should result in 15 rows for both rows. However the left join results in the first query for the first row are null and to my understanding ignored. I'll attach the query plans for both queries.
> I should note that there is a one to many relation between infection and admission so therefore infectionid is for the same admission.
> Strangely enough if you enclode the first query in a group by query and count the rows it does indeed return 2 times 15 for the specific groups (see enclosed_queryplan.txt).
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4622) Adding new data quality functions from osDQ
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4622?page=com.atlassian.jira.plugin... ]
Ramesh Reddy reassigned TEIID-4622:
-----------------------------------
Assignee: Kylin Soong (was: Steven Hawkins)
> Adding new data quality functions from osDQ
> -------------------------------------------
>
> Key: TEIID-4622
> URL: https://issues.jboss.org/browse/TEIID-4622
> Project: Teiid
> Issue Type: Feature Request
> Components: Common
> Affects Versions: Open To Community
> Environment: osdq-core and its dependency simmetric 3.0.0 is needed to run these functions
> maven central location
> http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22osdq-core%22
> Reporter: vivek singh
> Assignee: Kylin Soong
> Fix For: Open To Community
>
> Attachments: DateUtil.java, NumberUtil.java, StringUtil.java
>
>
> Following functions have been added into osdq-core for integration with teiid.
> String functions :
> // Encrypting a string using key and AES encryption
> String encrypt ( String val, String key)
> String decrypt ( String val, String key)
> // Fuzzy distance - similarity between two strings
> float levestheindistance ( String a, String b)
> float jarowinklerdistance( String a , String b)
> float cosinedistance (String a, String b)
> // Format check for PII ( personal information)
> boolean isCreditCardFormat(String a)
> boolean isSSNFormat(String a)
> boolean isPhoneFormat(String a)
> boolean isEmailFormat(String a)
> boolean hasWhiteSpace(String a )
> Date Functions
> // converting date into universal recognized format
> long dateToEpoch(java.util.Date date)
> Date epochToDate(long epoch)
> Date epochToDate(long epoch, String TimeZone)
> long diffInSec(date a, date b)
> String convertToFormat ( Date a, String Format)
> Numerical Functions
> // Statistical functions
> double stdDev(Number[] numSeries)
> double range(Number[] numSeries)
> double median(Number[] numSeries)
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years
[JBoss JIRA] (TEIID-4441) Add FTP/SFTP based access to the File Resource Adapter
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-4441?page=com.atlassian.jira.plugin... ]
Ramesh Reddy edited comment on TEIID-4441 at 12/5/16 2:44 PM:
--------------------------------------------------------------
One suggestion I have is do not download the file from FTP server and create a temp file locally, consider that as a stream of data. The metadata exposed from file translator already returns either Clob or Blob, so use those interfaces to pass in the contents of a file. Consider, when we do the same with HDFS this could be multi GB data at best or even larger, we should not to be making copies of these files.
Also make sure VFS is not dragging in any dependencies other than what you are defining, these need to be concise.
was (Author: rareddy):
One suggestion I have is do not download the file from FTP server and create a temp file locally, consider that as a stream of data. The metadata exposed from file translator already returns either Clob or Blob, so use those interfaces to pass in the contents of a file. Consider, when we do the same with HDFS this could be multi GB data at best or even larger, we should not to be making copies of these files.
> Add FTP/SFTP based access to the File Resource Adapter
> ------------------------------------------------------
>
> Key: TEIID-4441
> URL: https://issues.jboss.org/browse/TEIID-4441
> Project: Teiid
> Issue Type: Enhancement
> Components: Misc. Connectors
> Reporter: Ramesh Reddy
> Assignee: Kylin Soong
> Fix For: Open To Community, 9.2
>
>
> This has been requested few times in the community, it would be nice to give provide this functionality. Using http://commons.apache.org/proper/commons-net/ seems to be right decision.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years