Hi Max, Thanks for the responses. Find answers to your questions inline below. Cheers PGR Currently, 8.0 development uses * org.jboss.ide.eclipse.as.serverAdapter.wtp.feature.source.feature.group for access to jboss as7 API and framework. So according to the TP wizard, this comprises 18 plugins Rationalising the teiid jar dependencies, I have noticed that I can depend the teiid_embedded_query plugin on the following. This removes all jboss.as dependencies except for jboss-as-cli.jar * org.jboss.ide.eclipse.as.management.as71 * jcip-annotations.jar NO * saxon9he.jar NO * poi.jar, poi-scratchpad.jar, poi-contrib.jar NO * jaxen-core.jar, jaxen-jdom.jar, saxpath.jar NO * mockito-core.jar, com.springsource.org.objensis.jar YES/NO - Version in use is 1.9.1 but version 1.8.4 available. Only recently upgraded to 1.9.1 to use 'latest version' so not an issue to downgrade for the moment * wsdl4j.jar YES/NO - Version in use is 1.4 but both versions 1.5.1 and 1.6.2 are available With the release of teiid 8, backward compatibility has been broken so I would envisage a teiid versioned features being included and clients import specific versions of the feature/plugins using eclipse plugin functionality, eg. [8.0.0,9.0.0). Going forward, TD 8.0 will be the first application to use teiid 8 plugins and thus this P2 repository so we can set the rules for use? Correct. See my rationalising commment above for reduction of the jboss-as dependencies. If jboss-as-cli could be offered in a plugin then this need not be included either. At the moment, yes. It seems that the teiid jars are slowly splitting into constituent components, eg. admin-api, common-core, client, engine, but this seems ongoing. Not sure ... will gather details as I make progress ;) <snip> -- Paul Richardson * p.g.richardson(a)phantomjinx.co.uk * p.g.richardson(a)redhat.com * pgrichardson(a)linux.com "I know exactly who reads the papers ... * The Daily Mirror is read by people who think they run the country. * The Guardian is read by people who think they ought to run the country. * The Times is read by people who do actually run the country. * The Daily Mail is read by the wives of the people who run the country. * The Financial Times is read by the people who own the country. * The Morning Star is read by the people who think the country ought to be run by another country. * The Daily Telegraph is read by the people who think it is." Jim Hacker, Yes Minister

This *IS* weird, and I suspect it is not the correct way to do things. Almost no plugins should ever depend directly on as.management.as71, and by almost, i mean exactly none. They may require the feature (and plugin) be present, but they should have no compile time dependencies against the plugin. The correct way to do this is to depend on org.jboss.ide.eclipse.as.dmr. This is a simple wrapper for the DMR jboss-as logic which allows you to build remote management requests. Unfortunately, due to classloading issues, we can't just give the caller a ModelNode reference. Actually we can, right now, but if AS-8 or any other version of AS ever requires a new management api (and I am QUITE sure they will), this will break 100 different ways. So, the expected thing you should be doing is to depend on org.jboss.ide.eclipse.as.dmr, and do something like the following: private String createResourceRequest() { ModelNode request = new ModelNode(); request.get(OP).set(READ_RESOURCE_OPERATION); // blah blah blah make more model node request return request.toJSONString(true); } Once it's available as a JSON string, you should do the following: String resultJSON = JBoss7ManagerUtil.executeWithService(new JBoss7ManagerUtil.IServiceAware<String>() { public String execute(IJBoss7ManagerService service) throws Exception { return service.execute(new AS7ManagementDetails(getServer()), createResourceRequest()); } }, getServer()); This will require you to have org.jboss.ide.eclipse.as.core on your classpath. The as.core plugin will search for a service interface that fits the given version of the server (ie for a while we had two different implementers, one for as7 and one for as7.1. Appserver team eventually fixed the broken backwards compatability and so the other was removed.) If you guys are doing something that doesn't involve wtp IServer objects at all, and can't use the constructor for AS7ManagementDetails, you can always fill out the interface in your own way via IAS7ManagementDetails I'll try to get a wiki page up to demonstrate this, and maybe an example plugin if possible.

Is Teiid / teiid designer used against remote servers? Or is it all against local ones? One of the strategies I used when initiating our jmx tooling was that, since each version of jboss-as would require different client jars, I'd simply find the IRuntime, and load the jars onto the classpath right at that moment. I'd use a URL Classloader. In the case of remote servers, I simply demanded a copy of the server also be local so that I can at least load the client jars. Assuming you only need the client jars during critical operations, these jars don't need to be on the classpath at all times. In fact, having them on the CP at all times means you're bound to have your toolset work against only one server version at a time, which is clearly bad ;) On 10/09/2012 12:33 AM, phantomjinx wrote:

why ? how often are we talking and can't these operation be done in "bulk" ? note, we do this in hibernatetools and its for sure not pretty and we are leaning towards moving these calls out to an external vm to avoid the runtime version coupling. All depends how complex its interaction is - if just "deploy/undeploy" then it shouldn't be that hard.

On 10/08/2012 12:40 PM, Max Rydahl Andersen wrote: No. The Admin API has changed to such a degree that it is not backward compatible. I believe this is due to teiid 7.x targetting jboss 5.1 whilst teiid 8.x target jboss 7+. Barry, please clarify this if I am incorrect. Don't want to spread FUD unnecessarily. My thoughts were this would be a P2 repo with multiple versions of the teiid feature. It would be up to clients to set their plugin dependencies to the appropriate version. This is already done currently in the TD plugin codebase on the basis of teiid_embedded_query [8.0.0,9.0.0). Would this work? PGR -- Paul Richardson * p.g.richardson(a)phantomjinx.co.uk * p.g.richardson(a)redhat.com * pgrichardson(a)linux.com "I know exactly who reads the papers ... * The Daily Mirror is read by people who think they run the country. * The Guardian is read by people who think they ought to run the country. * The Times is read by people who do actually run the country. * The Daily Mail is read by the wives of the people who run the country. * The Financial Times is read by the people who own the country. * The Morning Star is read by the people who think the country ought to be run by another country. * The Daily Telegraph is read by the people who think it is." Jim Hacker, Yes Minister

On 10/08/2012 01:03 PM, Max Rydahl Andersen wrote: This is a problem that goes wider than the P2 repo. At the moment, TD 8.0 development has gone down this road: TD 8.0 -> Teiid 8+ -> jboss 7+ Thus, upgrading TD features to 8.0 means users will have to upgrade to teiid 8 and jboss 7. They will lose access to teiid 7 and jboss 5.1 existing installations. We have had discussions in TD about trying to bundle both teiid 7 and teiid 8 jars but its a scary and very problematic solution. Barry, can expand further on this if required. Cheers PGR -- Paul Richardson * p.g.richardson(a)phantomjinx.co.uk * p.g.richardson(a)redhat.com * pgrichardson(a)linux.com "I know exactly who reads the papers ... * The Daily Mirror is read by people who think they run the country. * The Guardian is read by people who think they ought to run the country. * The Times is read by people who do actually run the country. * The Daily Mail is read by the wives of the people who run the country. * The Financial Times is read by the people who own the country. * The Morning Star is read by the people who think the country ought to be run by another country. * The Daily Telegraph is read by the people who think it is." Jim Hacker, Yes Minister

btw. we kind of already have an "orbit" site for eclipes plugins: http://download.jboss.org/jbosstools/updates/requirements/ its not a single site per-se but contains subsites for the appropriate bits (putting them in one big one would be bad to use with Tycho since then *everything* in here would be considered the target platform). What is my biggest concern is though how often your dependencies actually needs updating ? On my first look I see these as "true" stable dependencies: And then this looks like something very often updated runtime component which is better managed within teiid project similar to how we got a org.jbosstools.forge.runtime bundle in Forge to allow bundling a specific runtime for ease-of-use. Two additional questions from me after thinking about this over the weekend is: 1) What is "com.springsource.org.objensis.jar" ? why is this osgi jar in the mockito plugin and not the original jar from http://code.google.com/p/objenesis/downloads/list ? 2) if these are not just the original jar rebuilt for osgi should we not be giving these unique names ? i.e. org.jbosstools.orbit.org.jaxen or similar ? (just like springsource does for their own repackaging?) /max On 05 Oct 2012, at 18:35, Max Rydahl Andersen <max.andersen(a)redhat.com&gt; wrote: >> Teiid Designer (TD) comprises part of jboss tools and provides an eclipse UI client interface for >> modelling and deploying virtual databases (vdbs) to a teiid server, which in turn is deployed in jboss. >> >> I have begun a discussion with mistria this afternoon on the possibility of a 3rd party P2 plugin >> repository that would serve as part of the Teiid Designer target platform. > > ok, cool - similar discussions are going on for how we handle this for soa-tooling in general/per part. > > Optimally we would like to avoid having a custom P2 repo for every release X every component, but we need to start somewhere :) > >> >> Background: >> >> TD (unreleased version 8) uses a target platform of: >> * http://download.jboss.org/jbosstools/updates/development/juno (JBT milestone build) >> * http://download.jboss.org/jbosstools/updates/juno/SR0/ (Eclipse Juno Target Platform) >> >> Thus, TD is a consumer of both the jbosstools TP and jbosstools features. > > okey - would be cool to know which features TD is using (just out of interest to understand what API's are actually used - remember JBT dont really have any API's outside AS tools currently) > >> However, contained in the current TD codebase are the following wrapped plugins, ie. wrapping >> 3rd-party built/released jars: >> > > How many of these are already in eclipse orbit ? > > The teiid_embedded_query sounds like something that are very version specific ? Does this > have to be matching the target server still ? > > Do I read it right above that teiid_embedded_query is simply wrapping those jars and only exposing limited API ? > i.e. commons-cli.jar, jboss-as-cli.jar does sound like things that easily could come problematic if exposed to rest of the eclipse plugin stack. > >> All of these jars are obtained from external websites, including the teiid jars (sf.net/projects/teiid). >> >> Problem: >> >> Everytime, an update of these jars is required, the new set of jars must be downloaded, included in >> the TD codebase and committed to the TD repository. > > Yes, you are basically updating the Target Platform. > >> Since we fetch the jbosstools and eclipse plugins from the target platform, it would be advantageous >> if these wrapped-plugins could be broken out into their own repository, ideally with no jars >> actually in the code repository then automatically built and served as features in a P2 repository. >> This P2 repository is then used by TD developers and TD build infrastructure. > > Yes - a Teiid targetplatform, optimally one that have these jars unified with the other "ontop" tooling set > so we know these can be installed together. > >> >> For example, with the teiid jars, a new release is uploaded to sf.net by the teiid team, downloaded >> by TD dev, fully wrapped as plugin/plugins and added to the P2 repository. > > hmm - do you really put a full unique teiid jar into the teiid plugin release s? > >> Discussion so far: >> >> * Mistria introduced me to the maven-dependency-plugin:get goal that could be executed during the >> generate-resources phase to download release jars build-time > > k > >> * Mistria also pointed out his new maven plugin that can take several platforms as input and produce >> a single target file > > hmm...which is that ? :) > >> * We agreed that this sounds very much like a jboss version of Eclipse Orbit and considered the use >> of this repository across all jboss projects rather than just TD (an external deps + JBoss.org stuff >> packaged as OSGi bundle in a p2 repo) > > Yes, similar idea have come up before. > > I see JBoss Tools Orbit as something else than a Target Platform - Orbit would be actual input to the actual target platform and what is later pointed to for updatesites. > >> * Unlike Eclipse Orbit, we would wrap existing release jars rather than rebuild project from source > > well, depends on the jars I guess - if Eclipse Orbit already have something rebuilt we could consider using that - > but that all depends about matching jars/apis. > >> * So a suggestion was made that given a project X normally has additional X.tools and/or X.designer >> projects, the latter should be responsible for shipping X as OSGI bundles > > What worries me is the "shipping as osgi bundles" since that is *not* trivial if we start exposing stuff that > becomes runtime version dependent - just something to be extremely careful about. > > its the reason why we refrained from re-bundling dmr.jar/as7 client library as osgi libraries since we needed to be sure > our Server tools could use multiple version of this at the same time from the tools. > > Just something to consider. > >> * Need approval since this represents an independent project. However, I would like to limit it at >> first to being a TD focused project until either other's find a use for it and/or it proves itself >> useful > > well, we already have plenty of jars that are candidates for this (i.e. mockito.jar) - my main concern is the mix of runtime aspects into tools as osgi > bundles since it can limit our ability to be multi-runtime compatible. > > *that said* i'm all for the creation of a JBoss Tools Orbit project in which we can add content after review that can be shared > and properly versioned. > >> * Consider a small independent P2 update site that is built manually and locally at first then move >> to automation when that is working > > yes, sounds cool - create it as a standalone github project and we can go to set that up. > >> Questions for getting started: >> >> * The hosting location for this P2 update site. Do I / Is it possible to have access to >> download.jboss.org to upload such a site? > > yes - I don't see a problem for that - we just need to figure out the rules/conventions for how this JBoss Tools Orbit should work. > > i.e. it would be anything but eclipse plugins. The jars can't modify overall system state (i.e. versions of jboss-modules or was it logging changed system behavior that might be good for one plugin but terrible for others). > > But yes, Definitely a big +1 for creation of a JBoss Tools Orbit kind of repo. > > /max >

This doesn't work unless these libraries work/obey with osgi class loading rules and don't need to i.e. load classes from your plugins to work (i.e. one of the reasons why we cant put hibernate jars into individual bundles is that hibernate dont yet work scalable/nicely in osgi). But again, my biggest problem is if we continue architecting the tools and runtime we build to be tying users to install a new version of the tools for each runtime he uses. what I don't grok is why our eclipse plugins needs jline.jar, jboss-as-cli.jar etc. These also overlap with what is Forge tools for example - but here there are a reason - i.e. it exposes the CLI approach. Yes in the repository, but that does not help our users using multiple versions of the runtime. okey. Yes, it qualifies for a rename since you/we are not the "owner" of that package/bundle name. Same reason why springsource adds com.springsource.* to the bundles to not collide plus to signal that these bundles are something else than the default ones. Why Eclipse Orbit dont do the same i've never understood but the fact they do that it means i.e. once they update to org.mockito 1.9 we would have a similar named and versioned bundle that would be something different (thats bad) /max

okey - thats fine - if the plugin is considered an "repackaging" of teiid-embedded.zip then it makes sense. But if you want to create something else where it is osgi compatible then that is not the case. but teiid-admin.jar is for running some client app on the shell - not needed by the tools or ? I think you are missing my point ;) We deliberately avoid exposing runtime stuff as osgi jars because A) the jars often don't work well in osgi land or even an eclipse plugin world (i.e. anything that loads native code, i.e. jline.jar or custom classes i.e. logging/spring/cdi/hibernate etc.) B) having tools plugins depend on runtime jars hardens their dependency on runtime versions making it very userunfriendly (i.e. users dont normally need to install a new and old IDE to work with Java 6 and 7 at the same time). It might be the case for teiid that this cannot be avoided but then Teiid needs to be packaged/distributed in a way that allows easy installation and uninstallation to switch between these versions IMO. /max

Yes, for the teiid runtime parts I believe this fits best the model we currently got on Forge. see http://anonsvn.jboss.org/repos/jbosstools/trunk/forge/plugins/org.jboss.t... and its pom.xml for this. The remainder plugins seem to be different "beasts", i.e. the mockito jar is not for the actual release site, but just for tests, correct? so it uses the CLI utility code - not the actual jline stuff (at least that is how it seems) yeah I know - Its been a long standing request Teiid Designer would stop being so version dependent - I was hoping Teiid 8 would at least made *some* improvements on this. Yes agreed - but seems Teiid at is core depends on something that enforces this... /max

just to be clear here: If teiid designer team decides that all the jars within the teiid runtime distribution should be distributed as a set of osgi bundles ('create something else' than just a plugin with a set of jars for the runtime) then the following should be considered: 1) Can these jars actually run/work if loaded within eclipse OSGI ? 2) Will the bundle names conflict with possible other teams distributing these osgi jars ? Then better prefix them to avoid unwanted collisions (this is similar to you why you should not be releasing your own versino of mockito into the maven repository under the same name as whoever manages the mockito libraries) 3) does it have any benefits ? (i.e. if these jars are best considered closed/local to teiid designer then it might just be much simpler to stay with one jar. but this of course have challenges if/when you need to coexist with other libraries that might include/declare similar packages - i.e. if this teiid jars exports org.jline.* stuff - it would conflict with those who might decide to export org.jline from a real jline library. Its all a balance act but some common sense and careful dependency management should be able to allow us to navigate it. /max