[JBoss JIRA] Commented: (TEIID-143) Query should 'find' connector types and jars rather than have them predefined in the config.
by Greg Haber (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-143?page=com.atlassian.jira.plug... ]
Greg Haber commented on TEIID-143:
----------------------------------
I'm not very invested in how the process of getting connector bundles and/or their associated jars onto a server works, but I do think it is important that we keep some method of hot-deploying either updated connector type bundles or individual jars. We really need to keep the tasks that would require a full server restart to an absolute minimum. This is because Teiid/JBEDSP servers are often a shared resource between multiple different user populations (each with their own VDB and possibly connector bindings or types) so it is painful to require a full server restart for a change in support of only one VDB.
This is even more of an issue when talking about embedded - as we are sharing a JVM with other applications, so the pain of a server restart becomes even greater.
Yes, in production this work is all done in maintenance windows, so there it isn't such an issue, but in development (and sometimes also test) environments usually you don't bother with windows.
OSGI bundles are hot deployable, so that would be a very suitable approach here. Before going too far down the OSGI path though, please make sure that this won't introduce problems using an embedded Teiid under our sibling project, JBoss App Server - I believe that their OSGI bundle support is still under development.
> Query should 'find' connector types and jars rather than have them predefined in the config.
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-143
> URL: https://jira.jboss.org/jira/browse/TEIID-143
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 6.0.0
> Reporter: John Doyle
> Fix For: 6.1.0
>
> Original Estimate: 1 week
> Remaining Estimate: 1 week
>
> Currently, dqp as delived with Designer is preconfigured with connector types. I think it should discover the connectors that are available upon startup. This would make it easier to add connectors to a kit. The current process for adding a connector to a kit is manual and error prone. We have problems with it on nearly every release.
> This is also an attractive feature for embedding query in other environments.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 9 months
[JBoss JIRA] Commented: (TEIID-143) Query should 'find' connector types and jars rather than have them predefined in the config.
by Ramesh Reddy (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-143?page=com.atlassian.jira.plug... ]
Ramesh Reddy commented on TEIID-143:
------------------------------------
Or simply you could use DNA(http://jboss.org/dna), that will do the version of the jar. That may be the direction in the future.
I want ask the basic question is how important is to support the connectors though extension modules? Is it a common practise in customer sites that they apply a patches to currently running server, instead of in a maintenance window? if we do not support connector jars in extension modules, and only support them as the physical resources on the disk, does that impact the support?
The advantage here is, none of the third party jars needed by a connector need to be defined as extension modules, thus not exposed to to whole system.
The more I think about this, these need to be OSGI (http://www.osgi.org/Main/HomePage) bundles. Java 7 is also introducing a module system (JSR 294, http://jcp.org/en/jsr/detail?id=294), however this at least a year away. OSGI helps us with the designer too, as finally connector bundles could be seen as the sources, and UI can built around in addition to that bundle. Also, tooling for this good in eclipse. The cons is more work in Teiid to make it support OSGI.
> Query should 'find' connector types and jars rather than have them predefined in the config.
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-143
> URL: https://jira.jboss.org/jira/browse/TEIID-143
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 6.0.0
> Reporter: John Doyle
> Fix For: 6.1.0
>
> Original Estimate: 1 week
> Remaining Estimate: 1 week
>
> Currently, dqp as delived with Designer is preconfigured with connector types. I think it should discover the connectors that are available upon startup. This would make it easier to add connectors to a kit. The current process for adding a connector to a kit is manual and error prone. We have problems with it on nearly every release.
> This is also an attractive feature for embedding query in other environments.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 10 months
[JBoss JIRA] Commented: (TEIID-143) Query should 'find' connector types and jars rather than have them predefined in the config.
by Larry O'Leary (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-143?page=com.atlassian.jira.plug... ]
Larry O'Leary commented on TEIID-143:
-------------------------------------
I do see the concern with keeping them in-sync but I can't imagine that it could be that difficult. You have a few options the say I see it:
* Directory contents change triggers an extension module scan
* Server start-up triggers an extension module scan
Extension Module Scan:
* Server performs a scan on the directory to get a list of extension names
* The list is then compared to already loaded extension modules
** If directory list contains an entry that does not exist in DB extension modules, load the missing extension from the directory
** If directory list contains an entry that already exists in the DB extension modules, use a diff scheme to determine if the extension should be replaced
The diff scheme could be a simple file date/time compared to extension module modify date/time. This would require us to use the original modification date and time from the file when an extension is imported. Time zone would also have to be taken into consideration.
Alternatively, a hash could be generated on the file as it is imported into the DB. When a file in placed in the extension modules directory that has the same name as an already loaded extension module the hash from the DB is used to compare to the hash of the file to be imported. If they are different then we analyse the file / DB entry further to see if an update is needed.
We could also greatly reduce the synchronization issue by moving the file upon successful import to a sub-directory under the extension modules hot/auto-deploy directory. This way if the file is present on more than one server the two servers won't get stuck in a deploy/update tug-of-war in the event that the the two files are actually different. Although the modify date/time scheme would prevent such a problem.
> Query should 'find' connector types and jars rather than have them predefined in the config.
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-143
> URL: https://jira.jboss.org/jira/browse/TEIID-143
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 6.0.0
> Reporter: John Doyle
> Fix For: 6.1.0
>
> Original Estimate: 1 week
> Remaining Estimate: 1 week
>
> Currently, dqp as delived with Designer is preconfigured with connector types. I think it should discover the connectors that are available upon startup. This would make it easier to add connectors to a kit. The current process for adding a connector to a kit is manual and error prone. We have problems with it on nearly every release.
> This is also an attractive feature for embedding query in other environments.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 10 months
[JBoss JIRA] Commented: (TEIID-143) Query should 'find' connector types and jars rather than have them predefined in the config.
by Ramesh Reddy (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-143?page=com.atlassian.jira.plug... ]
Ramesh Reddy commented on TEIID-143:
------------------------------------
The concern with file based loading is, server keeps a copy of the jars in the DB. In cluster situation they can get out of sync.
If connectors does not need DB based extension class loading, we can make this this work. i.e. Connector are loaded during the startup, every cluster member defines their own connectors and their dependencies in their installation directories. This may be a good thing. However to patch a connector you would require manually copy the jar to each cluster member. Patching though Console or Admin api would be cumbersome, as they not be dealing same version of the Connector.
Another requirement I should add is
6) Package them easy enough to expose them as plugins in the eclipse, so that they can be used as sources for metadata importing and as well as use under Embedded execution in Designer as connectors.
> Query should 'find' connector types and jars rather than have them predefined in the config.
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-143
> URL: https://jira.jboss.org/jira/browse/TEIID-143
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 6.0.0
> Reporter: John Doyle
> Fix For: 6.1.0
>
> Original Estimate: 1 week
> Remaining Estimate: 1 week
>
> Currently, dqp as delived with Designer is preconfigured with connector types. I think it should discover the connectors that are available upon startup. This would make it easier to add connectors to a kit. The current process for adding a connector to a kit is manual and error prone. We have problems with it on nearly every release.
> This is also an attractive feature for embedding query in other environments.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 10 months
[JBoss JIRA] Commented: (TEIID-143) Query should 'find' connector types and jars rather than have them predefined in the config.
by Larry O'Leary (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-143?page=com.atlassian.jira.plug... ]
Larry O'Leary commented on TEIID-143:
-------------------------------------
I like the idea of a CAF file containing everything needed for the connector. In the instances that the driver JARs can not be bundled into the CAF due to licensing issues then the CAF will still be loaded but unavailable for use. This would be much like SQuirreL does. If the driver can't be loaded then the driver definition is "disabled". We already have the concept of extension modules so if I install a CAF that does not have the needed drivers the default driver classpath will reference the driver JAR as an extension module and I will need to manually load the driver JAR into the extension modules.
I would recommend that we support hot-loading of extension modules via a configurable file system path. CAF(s) could be loaded in the same manner and maybe should be considered extension modules in themselves.
For example, if I have a set of connector types which are for the DataDirect drivers in a CAF file without the DataDirect driver JAR(s), I would simply drop the CAF and required JAR(s) into /application/lib/ext and when I went to define a new connector binding I would see the new DataDirect specific connector types in the drop-down list.
If on the other hand I installed the CAF file without the required JAR(s), the connector types wouldn't be visible on the drop-down list because the drivers could not be loaded. As an advanced configuration option there would be a check-box or a connector type edit view that would allow me to see all connector types regardless if their drivers could be loaded as a means of giving me the ability to modify the connector type configuration to point it to a new JAR or a new driver class.
> Query should 'find' connector types and jars rather than have them predefined in the config.
> --------------------------------------------------------------------------------------------
>
> Key: TEIID-143
> URL: https://jira.jboss.org/jira/browse/TEIID-143
> Project: Teiid
> Issue Type: Feature Request
> Components: Misc. Connectors
> Affects Versions: 6.0.0
> Reporter: John Doyle
> Fix For: 6.1.0
>
> Original Estimate: 1 week
> Remaining Estimate: 1 week
>
> Currently, dqp as delived with Designer is preconfigured with connector types. I think it should discover the connectors that are available upon startup. This would make it easier to add connectors to a kit. The current process for adding a connector to a kit is manual and error prone. We have problems with it on nearly every release.
> This is also an attractive feature for embedding query in other environments.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 10 months
[JBoss JIRA] Resolved: (TEIID-66) dqp lifecycle issues
by Ramesh Reddy (JIRA)
[ https://jira.jboss.org/jira/browse/TEIID-66?page=com.atlassian.jira.plugi... ]
Ramesh Reddy resolved TEIID-66.
-------------------------------
Resolution: Done
> dqp lifecycle issues
> --------------------
>
> Key: TEIID-66
> URL: https://jira.jboss.org/jira/browse/TEIID-66
> Project: Teiid
> Issue Type: Bug
> Components: Embedded
> Affects Versions: 6.0.0
> Reporter: Steven Hawkins
> Assignee: Ramesh Reddy
> Fix For: 6.0.0
>
>
> There are some exceptions (see below) happening during Embedded shutdown. The CacheProvider looks like it's assuming that it's a true singleton, but the EmbeddedGuiceModule is created and the Injector is set on the ResourceFinder with ever EmbeddedConnection created. That's causing us to attempt the mbean registration and cache creation with every connection, which seems very troubling with simultaneous connections. What do you think about the following:
> 1. Why should we initiate an implicit shutdown with no connections? Why not implicitly start, but require an explicit shutdown.
> 2. We should at least promote the set of the injector to the constructor of EmbeddedConnectionFactoryImpl (assuming 1 jboss cache per dqp), however I don't know enough about how we're using JBoss cache to know if that will work. It seems like we would need to use a cache region per dqp, not a separate cache. But if it is one cache per, then at the very least we would need to also change our mbean registration to have unique names.
> 3. It looks like we can avoid the exception below if we first unregister the existing instance. Logically we'd like that to happen when JBossCacheFactory.destroy is called, so I'd be in favor of moving the mbean registration and dergistration logic into the JBossCacheFactory so that it has control over both sides.
> > javax.management.InstanceAlreadyExistsException: jboss.cache:service=MetaMatrixCacheTree
> > at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
> > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
> > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
> > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
> > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
> > at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
> > at com.metamatrix.jdbc.CacheProvider.get(CacheProvider.java:51)
> > at com.metamatrix.jdbc.CacheProvider.get(CacheProvider.java:40)
> > at com.google.inject.BoundProviderFactory.get(BoundProviderFactory.java:60)
> From Ramesh:
> 3. Yes, this needs to be done.
> 2. DQP only uses the cache for the resultset caching. I am kind of on
> the fence about supporting multiple DQP instances, in a given VM. I do
> not see that as serving much of the usecases. So, if we can relax that
> requirement then we can always assume there is only one instance of the
> DQP. Otherwise, depending upon how we use Cache we should be able to
> segment cache by DQP instance identifier.
> 1. Except for re-loading times, do you see any issue with this? Original
> design wanted to see the Embedded as JDBC driver, such that when there
> are no connections, it unloads itself and reduces the memory footprint
> of the VM. It become more of who's responsibility is it to shutdown? If
> we deployed this app server, or in pre-built tool like Squirrel.
> Shutting down closes the connector bindings etc, that are active.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
15 years, 10 months