[JBoss JIRA] (TEIID-3632) Performance issue retrieving data source and translator definitions through the AdminFactory
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-3632?page=com.atlassian.jira.plugin... ]
Ramesh Reddy resolved TEIID-3632.
---------------------------------
Labels: Beta2 (was: )
Resolution: Done
Made couple more changes
- For the calls getInstalledResourceAdaptorNames(), getConnectionFactoryNames(), getDeployedResourceAdaptorNames() introducing a time based expiring cache object that lives for 5 secs. So, any admin calls made inside that 5 secs use the cached objects rather then re-issuing the query. I do not go any lengths to remove these objects after 5 secs, rather I mark them as expired. If they are expired another set of fresh CLI calls will be made transparently when next time they are accessed.
- Introduced "ALL" TranslatorPropertyType, using which one can ask for all the IMPORT, OVERRIDE, EXTENSION properties on the getTranslatorPropertyDefinitions() method. The returned properties will have a new attribute called "category" which will define the type of the property IMPORT, OVERRIDE, EXTENSION
> Performance issue retrieving data source and translator definitions through the AdminFactory
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-3632
> URL: https://issues.jboss.org/browse/TEIID-3632
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Reporter: Barry LaFond
> Assignee: Ramesh Reddy
> Labels: Beta2
> Fix For: 8.12
>
> Attachments: server_refresh_cli_log.txt
>
>
> Based on the issue described in TEIIDDES-2592, we discovered that connecting to a remote server that contained ~ 40 data sources required > 10 minutes to refresh the Server view in Teiid Designer. After some debugging it appeared that some of the current behavior in AdminFactory results multiple redundant CLI calls to retrieve intermediate information to compile even a simple list of data source names.
> In Teiid Designer, we modified our forked Admin8Factory to define a few cached sets of data that get cleared if data is changed by our ExecutionAdmin class for any created or deleted data sources.
> Note that Designer's Server management paradigm includes the requirement to view the DS/Translator/VDB contents on the server.
> So I'm assuming that there could be some benefit if Teiid AdminFactory adapted a similar approach to improving the performance/behavior of the internal methods of this class.
> see: https://github.com/Teiid-Designer/teiid-designer/blob/master/plugins/teii... for details
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (TEIID-3642) RhinoScriptEngineFactory be removed in Java 8
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-3642?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-3642:
------------------------------------------------
Alex Szczuczko <aszczucz(a)redhat.com> changed the Status of [bug 1254399|https://bugzilla.redhat.com/show_bug.cgi?id=1254399] from ASSIGNED to MODIFIED
> RhinoScriptEngineFactory be removed in Java 8
> ----------------------------------------------
>
> Key: TEIID-3642
> URL: https://issues.jboss.org/browse/TEIID-3642
> Project: Teiid
> Issue Type: Quality Risk
> Components: OData
> Affects Versions: 8.7.1.6_2, 8.12
> Environment: * DV 6.2.0.ER4
> * Java 1.8.0_25
> Reporter: Kylin Soong
> Assignee: Steven Hawkins
>
> RhinoScriptEngineFactory be removed in Java 8, this cause OData war deploy output Error:
> {code}
> 09:30:51,315 WARN [org.jboss.as.dependency.unsupported] (MSC service thread 1-6) JBAS015868: Deployment "deployment.teiid-odata-8.7.1.6_2-redhat-2.war" is using an unsupported module ("org.joda.time:main") which may be changed or removed in future versions without notice.
> 09:30:51,424 ERROR [stderr] (MSC service thread 1-6) ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider com.sun.script.javascript.RhinoScriptEngineFactory not found
> 09:30:51,429 ERROR [stderr] (MSC service thread 1-1) ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider com.sun.script.javascript.RhinoScriptEngineFactory not found
> 09:30:51,505 ERROR [stderr] (MSC service thread 1-1) ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider com.sun.script.javascript.RhinoScriptEngineFactory not found
> 09:30:51,513 ERROR [stderr] (MSC service thread 1-1) ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider com.sun.script.javascript.RhinoScriptEngineFactory not found
> 09:30:51,583 ERROR [stderr] (MSC service thread 1-1) ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider com.sun.script.javascript.RhinoScriptEngineFactory not found
> {code}
> *OData war* depend on *org.joda.time*, *org.joda.time* depend on *RhinoScriptEngineFactory*, due to RhinoScriptEngineFactory be removed in Java 8, so stderr output in console.
> h3. how to reproduce
> Start DV 6.2 with Java 8.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (TEIID-3534) Infinispan-dsl-cache translator: can't insert null into column of type boolean
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-3534?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-3534:
------------------------------------------------
David Le Sage <dlesage(a)redhat.com> changed the Status of [bug 1232236|https://bugzilla.redhat.com/show_bug.cgi?id=1232236] from NEW to ASSIGNED
> Infinispan-dsl-cache translator: can't insert null into column of type boolean
> ------------------------------------------------------------------------------
>
> Key: TEIID-3534
> URL: https://issues.jboss.org/browse/TEIID-3534
> Project: Teiid
> Issue Type: Bug
> Components: Misc. Connectors
> Affects Versions: 8.7.1.6_2
> Reporter: Filip Elias
> Assignee: Van Halbert
> Attachments: server.log, testVDB.vdb
>
>
> Inserting null into boolean column causes exception.
> Sample queries:
> {code}
> insert into smalla(intKey, stringKey,booleanValue) values(128,'ss',null);
> insert into smalla(intKey, stringKey) values(128,'ss');
> {code}
> This query works:
> {code}
> insert into smalla(intKey, stringKey,booleanValue) values(128,'ss',false);
> {code}
> Exception:
> {code}
> Connector worker process failed for atomic-request=NuZ8Nt3h1bKx.52.0.44: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for message id[239] returned server error (status=0x85): org.hibernate.search.bridge.BridgeException: Exception while calling bridge#set
> class: org.infinispan.query.remote.indexing.ProtobufValueWrapper
> path:
> at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:298)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:88)
> at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:74)
> at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56)
> {code}
> This bug breaks data integrity. If this exception is thrown, data will be inserted but it won't be indexed.
> The whole exception is in the attachment.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (TEIID-3609) Teiid Connection import: driver field not populated for resource adapters
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/TEIID-3609?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on TEIID-3609:
------------------------------------------------
David Le Sage <dlesage(a)redhat.com> changed the Status of [bug 1246116|https://bugzilla.redhat.com/show_bug.cgi?id=1246116] from NEW to ASSIGNED
> Teiid Connection import: driver field not populated for resource adapters
> -------------------------------------------------------------------------
>
> Key: TEIID-3609
> URL: https://issues.jboss.org/browse/TEIID-3609
> Project: Teiid
> Issue Type: Bug
> Reporter: Andrej Šmigala
> Assignee: Ramesh Reddy
>
> When importing using Teiid Connection, the driver field is not populated for resource adapters when the pool name of the connection definition is the same as the id of the resource adapter itself. Except for the case that there is _another_ resource adapter defined for the same module _without_ any connection definitions.
> In other words, when the following is included in standalone.xml, the driver field is not populated
> {code:xml}
> <resource-adapter id="file">
> <module slot="main" id="org.jboss.teiid.resource-adapter.file"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.file.FileManagedConnectionFactory" jndi-name="java:/file" enabled="true" pool-name="file">
> <config-property name="ParentDirectory">
> /home/
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
> {code}
> However, everything works as expected when the configuration is changed to this:
> {code:xml}
> <resource-adapter id="file">
> <module slot="main" id="org.jboss.teiid.resource-adapter.file"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.file.FileManagedConnectionFactory" jndi-name="java:/fileDS" enabled="true" pool-name="fileDS">
> <config-property name="ParentDirectory">
> /home/
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
> {code}
> Or this:
> {code:xml}
> <resource-adapter id="file">
> <module slot="main" id="org.jboss.teiid.resource-adapter.file"/>
> </resource-adapter>
> <resource-adapter id="fileDS">
> <module slot="main" id="org.jboss.teiid.resource-adapter.file"/>
> <transaction-support>NoTransaction</transaction-support>
> <connection-definitions>
> <connection-definition class-name="org.teiid.resource.adapter.file.FileManagedConnectionFactory" jndi-name="java:/fileDS" enabled="true" pool-name="fileDS">
> <config-property name="ParentDirectory">
> /home/
> </config-property>
> </connection-definition>
> </connection-definitions>
> </resource-adapter>
> {code}
> This might be related to TEIIDDES-1895 and is most likely the original cause of TEIIDDES-2563.
> Clicking Next in the import wizard when such a datasource is selected causes the same error as described in TEIIDDES-2563.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (TEIID-3632) Performance issue retrieving data source and translator definitions through the AdminFactory
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3632?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-3632:
---------------------------------------
Made the simple change to consolidate getting the resource adapter info and corrected a message.
> Performance issue retrieving data source and translator definitions through the AdminFactory
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-3632
> URL: https://issues.jboss.org/browse/TEIID-3632
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Reporter: Barry LaFond
> Assignee: Ramesh Reddy
> Fix For: 8.12
>
> Attachments: server_refresh_cli_log.txt
>
>
> Based on the issue described in TEIIDDES-2592, we discovered that connecting to a remote server that contained ~ 40 data sources required > 10 minutes to refresh the Server view in Teiid Designer. After some debugging it appeared that some of the current behavior in AdminFactory results multiple redundant CLI calls to retrieve intermediate information to compile even a simple list of data source names.
> In Teiid Designer, we modified our forked Admin8Factory to define a few cached sets of data that get cleared if data is changed by our ExecutionAdmin class for any created or deleted data sources.
> Note that Designer's Server management paradigm includes the requirement to view the DS/Translator/VDB contents on the server.
> So I'm assuming that there could be some benefit if Teiid AdminFactory adapted a similar approach to improving the performance/behavior of the internal methods of this class.
> see: https://github.com/Teiid-Designer/teiid-designer/blob/master/plugins/teii... for details
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (TEIID-3632) Performance issue retrieving data source and translator definitions through the AdminFactory
by Ramesh Reddy (JIRA)
[ https://issues.jboss.org/browse/TEIID-3632?page=com.atlassian.jira.plugin... ]
Ramesh Reddy commented on TEIID-3632:
-------------------------------------
- Should be able to combine OVERRIDE, IMPORT and EXTENSION_METADATA for all template calls from 3 to 1
- Unless we introduce the concept of "context" or state into Admin, I am not sure how to avoid them as AS does not give an option say give me information on "all" resource-adapters in single call.
Another option is to pull in the management of the resource-adapters inside Teiid CLI and provide the necessary level of reduction of calls.
> Performance issue retrieving data source and translator definitions through the AdminFactory
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-3632
> URL: https://issues.jboss.org/browse/TEIID-3632
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Reporter: Barry LaFond
> Assignee: Ramesh Reddy
> Fix For: 8.12
>
> Attachments: server_refresh_cli_log.txt
>
>
> Based on the issue described in TEIIDDES-2592, we discovered that connecting to a remote server that contained ~ 40 data sources required > 10 minutes to refresh the Server view in Teiid Designer. After some debugging it appeared that some of the current behavior in AdminFactory results multiple redundant CLI calls to retrieve intermediate information to compile even a simple list of data source names.
> In Teiid Designer, we modified our forked Admin8Factory to define a few cached sets of data that get cleared if data is changed by our ExecutionAdmin class for any created or deleted data sources.
> Note that Designer's Server management paradigm includes the requirement to view the DS/Translator/VDB contents on the server.
> So I'm assuming that there could be some benefit if Teiid AdminFactory adapted a similar approach to improving the performance/behavior of the internal methods of this class.
> see: https://github.com/Teiid-Designer/teiid-designer/blob/master/plugins/teii... for details
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (TEIID-3632) Performance issue retrieving data source and translator definitions through the AdminFactory
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3632?page=com.atlassian.jira.plugin... ]
Steven Hawkins commented on TEIID-3632:
---------------------------------------
>From the log/code - admin.getDataSource("ldap1")
will get all ~22 resource-adapters 3 times: Once in getDataSourceNames/getConnectionFactoryNames, then getConnectionFactoryNames, and again in getResourceAdapterNames when the driver-name is null.
This is repeated for each data source, so yes there is an excessive number of calls. This could be consolidated to a single call per datasource, but there's still the issue that getDataSource for a single source will still iterate over all resource-adapters. Is there any way to avoid that?
> Performance issue retrieving data source and translator definitions through the AdminFactory
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-3632
> URL: https://issues.jboss.org/browse/TEIID-3632
> Project: Teiid
> Issue Type: Bug
> Components: AdminApi
> Reporter: Barry LaFond
> Assignee: Ramesh Reddy
> Fix For: 8.12
>
> Attachments: server_refresh_cli_log.txt
>
>
> Based on the issue described in TEIIDDES-2592, we discovered that connecting to a remote server that contained ~ 40 data sources required > 10 minutes to refresh the Server view in Teiid Designer. After some debugging it appeared that some of the current behavior in AdminFactory results multiple redundant CLI calls to retrieve intermediate information to compile even a simple list of data source names.
> In Teiid Designer, we modified our forked Admin8Factory to define a few cached sets of data that get cleared if data is changed by our ExecutionAdmin class for any created or deleted data sources.
> Note that Designer's Server management paradigm includes the requirement to view the DS/Translator/VDB contents on the server.
> So I'm assuming that there could be some benefit if Teiid AdminFactory adapted a similar approach to improving the performance/behavior of the internal methods of this class.
> see: https://github.com/Teiid-Designer/teiid-designer/blob/master/plugins/teii... for details
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months
[JBoss JIRA] (TEIID-3645) Upgrade of Amazon aws sdk version
by Steven Hawkins (JIRA)
[ https://issues.jboss.org/browse/TEIID-3645?page=com.atlassian.jira.plugin... ]
Steven Hawkins resolved TEIID-3645.
-----------------------------------
Resolution: Done
Updated master. There will likely be an 8.11.3, so we'll determine if this should go in there as well.
> Upgrade of Amazon aws sdk version
> ---------------------------------
>
> Key: TEIID-3645
> URL: https://issues.jboss.org/browse/TEIID-3645
> Project: Teiid
> Issue Type: Component Upgrade
> Components: Misc. Connectors
> Affects Versions: 8.11.2
> Environment: teiid 8.11.2
> windows 8.1
> java 8
> Reporter: avinash misra
> Assignee: Steven Hawkins
> Fix For: 8.12
>
>
> I was working on dynamodb connector with latest dynamodb sdk.
> This sdk uses aws-java-sdk-core version 1.10.11 but teiid uses aws-java-sdk 1.5.7. internally for simple db connector.
> Could you please upgrade the aws sdk version to latest?
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
10 years, 7 months