[jbosstools-dev] Regarding Stacks, Runtimes, and Remote Descriptors

[Rob Stryker]

Regarding Stacks, Runtimes, and Remote Descriptors

Hi All:

This email is to try to begin discussion on some recent duplication of 
code and responsibilities, which should probably be fixed before things 
get too comfortable.  I'm speaking specifically about the role of 
discovering runtimes to download, where that's done, how that's done, 
and which responsibility belongs to who. Forgive me if the email is 
long, as I am trying to be thorough.

Currently, there are three places from which runtimes to download may be 
discovered.

1) base/runtimes has an extension point named downloadRuntimes, which is 
used by AS Tools and Seam Tools (and perhaps others).

2) a remote descriptor file which acts as a second arm of 1) and is 
basically an xml form of 1) used to dynamically add new runtimes as they 
become available

3) The new Stacks methodology, currently stored in 
jbosstools-central/maven/plugins/org.jboss.tools.maven.project.examples

We should begin unifying these three locations into one, but the goal is 
to do it correctly. So, I would first like to list the benefits of each.

a) downloadable runtimes provided through the extension point cannot be 
removed without a maintenance or major release of some type, and for 
this reason are semi-permanent

b) downloadable runtimes available via the remote descriptor file may be 
added OR removed at will. This provides flexibility and post-release 
updates are easy.

c) The new stacks section has a more robust model capable of providing 
more information than the downloadable runtimes does. However, the 
plugin requires several libraries and is currently placed in the 
jboss-central module, where others may not make use of it.

d) the Stacks yaml file does not provide a place to access the file size 
for the download, however it does provide a 'labels' section, which 
seems extendable to add whatever properties you may want to add.

At first glance, it seems that Stacks is the superior framework. It is 
extensible, it can have unlimited labels (aka properties) if desired, 
and it already provides more information which is usable to others who 
may want it. To make use of stacks inside Runtimes, however, we'd either 
need to:
    a) Expand the API in runtimes to allow other plugins (like in 
central) to provide downloadable runtimes,  or,
    b) Push 'stacks' out of central and down into runtimes, as its own 
plugin upon which runtimes.core and runtime.ui can depend.

The main negative of pushing stacks into base/runtimes, in my opinion, 
is that there are a significant number of libraries required. It's not 
too much, by far, but it is about 7 jars totalling about 1 megabyte. 
Whether these jars belong in base/runtimes is debatable, and currently 
we do not have a "3rd-party dependencies" section in base where we 
organize common dependencies and versions together so that each plugin 
doesn't need to bundle their own 3rd party libraries. I admit, this is a 
debate for another time, but, I just wanted to point out that pushing 
the stacks logic down into runtimes would be another example of this issue.

Even still, I would argue that we should push stacks into its own small 
plugin below runtimes, deprecate the "downloadRuntimes" extension point, 
and the online downloadRuntime.xml (wherever the file is, I forget).

However, once we do that, there are many more questions. The first is, 
who's job is it to provide the yaml file from which stacks are generated?

Currently there is only one yaml file, and it is referenced directly via 
a github url. Aside from how (IMO) this is fairly crazy in itself, it 
causes another problem. The stacks client jar *can* cache the yaml file 
and only update if the timestamp has changed, however, when checking the 
timestamp on a github file, there isn't one...

This would seem to imply we should take control of the yaml file 
ourselves and put it NOT in github but rather in a release-specific 
online-accessible folder, ex: jbt4.1/stacks.yaml, jbt5.0/stacks.yaml, etc.

The problem with this is that we are then taking control away from the 
jdf team, and once we take the file away, it is our job to keep it 
updated and in synch. This may cause errors if we are not very careful.

Assuming we do this, though, the next question is, do we add seam and 
esb runtimes to this yaml file, which currently only provides 
application servers? Remember, the purpose of moving stacks down would 
be to deprecate the downloadRuntime extension point, therefore any 
replacement would need to do everything downloadRuntime does, which 
includes providing seam and esb runtimes for download.

Let's assume (for now) that we simply add lines to the yaml to allow it 
to provide seam and esb runtimes. We may come back to this point later, 
but for now, assume we do that.


Then which plugin will provide the url to our copied yaml? Who's 
responsibility is it to point to this yaml file? Let's look at our options:

1) The runtimes plugin references the yaml
2) The central plugin references the yaml

Both of these fail after thinking about it. How?

1) If the runtimes plugin references the yaml, then the download 
runtimes dialog will list things (like seam) which may not be present in 
the installation. Imagine an installation with only base and server 
plugins installed, and so no seam or esb. A user clicking 'download 
runtimes' will see esb and seam downloads, but the plugins which are 
prepared to handle those runtimes after the download are not present.

2) If central is in charge of providing this yaml, perhaps through a new 
extension point to the base/runtimes/stacks plugin we add there, then an 
installation including only plugins from base / server will have a BLANK 
download list. Users who install only ASTools will not be able to 
download JBoss Application Servers.

So both of these fail in their own way. The only solution as I can see, 
the only way it would work, would be to have multiple such yaml files, 
one for astools, one for seam, one for esb, etc. Each of these modules 
would provide their own yaml url to base/runtimes/stacks via an 
extension point in base/runtimes/stacks, and let stacks fetch each one 
and build a unified model.

Problems:
   a) multiple urls need to be loaded
   b) multiple yaml files need to be kept up to date, instead of just 
one. Multiply number of contributing plugins by number of major releases
   c) Possibility of duplicates. Once you have multiple yaml files 
generating models, it's possible some duplication leaks in. I'm not so 
sure about this one, but Fred listed it as a concern.

So, by my analysis, this is the only way I can imagine a unification of 
these three models. I'll summarize the changes below, but it does seem 
there would be a bit of work to do.

Summary of changes:
  1) Deprecate downloadRuntimes extension point
  2) Create new plugin in runtimes module called "stacks"
  3) Add extension point to 'stacks' plugin called stacksProvider
  4) modify runtime.core and runtime.ui to use the model built in 'stacks'
  5) Create a web-accessible location for jbt-release-relevent data on a 
per-module basis. For example, 
http://wherever/jbt/4.1.0/stacks/astools.yaml, etc.
  6) Copy the current jdf yaml file to that location for astools.yaml
  7) Create a new yaml file which can build stacks for esb, seam, etc
  8) Ensure astools, esb, seam, etc, make use of the new stacksProvider 
extension point
  9) Test the shit out of it

There are other benefits to this approach. Currently there's no really 
good mapping of downloadRuntimes id's to an app-server id. This is done 
in a hard-coded fashion in astools. This could instead be added to the 
labels in the astools.yaml file if desired. It would allow dynamic 
addition or removal of any runtimes, though in the yaml syntax. It would 
minimize connections and re-downloads of the yaml files, since they'll 
actually have a timestamp now (as opposed to in github, where they 
don't). And it could help clean up some other areas that could benefit 
from a cleanup.

I'd really like feedback on this issue from anyone who knows anything 
about the topic, because I know for sure I'm lacking a bit in fully 
understanding the entire api. But I'd love at the least for someone to 
tell me which of the logic here is obviously bad or if i'm wrong on any 
details.

Thanks and look forward to the feedback

- Rob Stryker
I break things, and then put them back together.