[jbosstools-dev] Regarding Stacks, Runtimes, and Remote Descriptors

Rob Stryker rstryker at redhat.com
Mon Apr 1 04:39:27 EDT 2013 

Hi Rafael:

I'm sure I and the tools team will have more questions in the future, 
but, right now one question I have is whether it's planned to add ALL of 
our runtimes to the stacks? Things like jpp, GateIn, SOA-P, etc?

So far it seems limited to AS and EAP.

Thanks!

- Rob Stryker

On 03/27/2013 12:39 AM, Rafael Benevides wrote:
> Hi Fred and all JBDS team!
>
> The idea of "merge data from separate yaml files" doesn't worked and 
> we (JDF team) didn't put any more effort on it.
>
> The original idea is that we could join the information (appending a 
> new content to the original stacks.yaml) and use its YAML anchors. To 
> clarify what I'm saying, look the matrix.yaml file 
> (https://github.com/jboss-jdf/jdf-stack/blob/1.0.0.Final/matrix.yaml) 
> it has some references to BOMs and archtypes that is defined on 
> original stacks.yaml.
>
> The main problem is that the new generarate yaml file is not parsable 
> via StacksClient (because it expects the original 1.0.0 format only) 
> and a new "client" should be built to each new format extension. This 
> could make us loose the control since any data change on stacks.uaml 
> could impact all extensions that uses it. So in fact: It was not a 
> good idea to have this extensions on separate files.
>
> The next "Stacks 1.0" planned evolution is 
> https://issues.jboss.org/browse/JDF-222 - Stacks should permit a 
> "Early Access" Runtimes - And because of the file format, this "Early 
> Acccess" will be a new "label" but I have afraid that consumers must 
> know that "allAvailableRuntimes" has "Released/Final" and "Early 
> Access" runtimes mixed. They should query the labels to distinguish them.
>
> That's my 2 cents.
>
>
> Em 26/03/13 10:25, Fred Bricon escreveu:
>> Hey Rafael,
>>
>> what's the status on the "merge data from separate yaml files"? I 
>> know you worked on that at some point.
>> We'll most certainly need to combine JBoss Tools specific runtimes or 
>> infos with the existing JDF ones.
>>
>> Fred
>>
>> Le 02/03/2013 08:52, Rob Stryker a écrit :
>>> license, size, and disclaimer can all be added in the labels section of
>>> the yaml.
>>>
>>> Max has already made it clear that having three things that do the same
>>> thing is wasteful and confusing, so the question isn't *if* we will
>>> unify them, but rather *how*.
>>>
>>> On 03/01/2013 09:29 PM, Snjezana Peco wrote:
>>>> I think we need to keep all of those options (since they already
>>>> exist). Each of them has its advantages/disadvantages.
>>>> I agree with the proposed changes and emphasize that stacks don't have
>>>> the following properties:
>>>>
>>>> - license
>>>> - size
>>>> - requireSso (see 
>>>> https://github.com/jbosstools/jbosstools-base/pull/50)
>>>> - disclaimer
>>>>
>>>> Snjeza
>>>>
>>>> On 3/1/2013 10:22 AM, Rob Stryker wrote:
>>>>> I would like to hear some feedback here from Max and Snjezana.
>>>>>
>>>>> On 02/28/2013 08:05 PM, Fred Bricon wrote:
>>>>>> I  mostly agree with the changes you described. Here's my 0.02€ :
>>>>>> - I strongly believe runtimes should be split into different stacks
>>>>>> descriptors, but I don't like the idea of having to maintain a 
>>>>>> fork of
>>>>>> the JDF one. We should only add extensions (old AS'es, seam, ESB 
>>>>>> ...).
>>>>>> - I believe the merge all stacks descriptors in one metamodel 
>>>>>> should be
>>>>>> done in stacks client. I know Raphael kinda started working on 
>>>>>> that a
>>>>>> few months back, he probably can give us his insight.
>>>>>> - runtimes in stacks could list their managing JBT plugins in the
>>>>>> labels property : i.e if ESB runtime can be downloaded, but no ESB
>>>>>> plugin is installed, the we'd be able to discover both the 
>>>>>> runtime AND
>>>>>> its associated plugin
>>>>>>
>>>>>> Fred
>>>>>>
>>>>>> Le mercredi 27 février 2013 18:22:35, Rob Stryker a écrit :
>>>>>>> Regarding Stacks, Runtimes, and Remote Descriptors
>>>>>>>
>>>>>>> Hi All:
>>>>>>>
>>>>>>> This email is to try to begin discussion on some recent 
>>>>>>> duplication of
>>>>>>> code and responsibilities, which should probably be fixed before
>>>>>>> things get too comfortable.  I'm speaking specifically about the 
>>>>>>> role
>>>>>>> of discovering runtimes to download, where that's done, how that's
>>>>>>> done, and which responsibility belongs to who. Forgive me if the 
>>>>>>> email
>>>>>>> is long, as I am trying to be thorough.
>>>>>>>
>>>>>>> Currently, there are three places from which runtimes to 
>>>>>>> download may
>>>>>>> be discovered.
>>>>>>>
>>>>>>> 1) base/runtimes has an extension point named downloadRuntimes, 
>>>>>>> which
>>>>>>> is used by AS Tools and Seam Tools (and perhaps others).
>>>>>>>
>>>>>>> 2) a remote descriptor file which acts as a second arm of 1) and is
>>>>>>> basically an xml form of 1) used to dynamically add new runtimes as
>>>>>>> they become available
>>>>>>>
>>>>>>> 3) The new Stacks methodology, currently stored in
>>>>>>> jbosstools-central/maven/plugins/org.jboss.tools.maven.project.examples 
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> We should begin unifying these three locations into one, but the 
>>>>>>> goal
>>>>>>> is to do it correctly. So, I would first like to list the 
>>>>>>> benefits of
>>>>>>> each.
>>>>>>>
>>>>>>> a) downloadable runtimes provided through the extension point 
>>>>>>> cannot
>>>>>>> be removed without a maintenance or major release of some type, and
>>>>>>> for this reason are semi-permanent
>>>>>>>
>>>>>>> b) downloadable runtimes available via the remote descriptor 
>>>>>>> file may
>>>>>>> be added OR removed at will. This provides flexibility and
>>>>>>> post-release updates are easy.
>>>>>>>
>>>>>>> c) The new stacks section has a more robust model capable of 
>>>>>>> providing
>>>>>>> more information than the downloadable runtimes does. However, the
>>>>>>> plugin requires several libraries and is currently placed in the
>>>>>>> jboss-central module, where others may not make use of it.
>>>>>>>
>>>>>>> d) the Stacks yaml file does not provide a place to access the file
>>>>>>> size for the download, however it does provide a 'labels' section,
>>>>>>> which seems extendable to add whatever properties you may want 
>>>>>>> to add.
>>>>>>>
>>>>>>> At first glance, it seems that Stacks is the superior framework. 
>>>>>>> It is
>>>>>>> extensible, it can have unlimited labels (aka properties) if 
>>>>>>> desired,
>>>>>>> and it already provides more information which is usable to 
>>>>>>> others who
>>>>>>> may want it. To make use of stacks inside Runtimes, however, we'd
>>>>>>> either need to:
>>>>>>>       a) Expand the API in runtimes to allow other plugins (like in
>>>>>>> central) to provide downloadable runtimes,  or,
>>>>>>>       b) Push 'stacks' out of central and down into runtimes, as 
>>>>>>> its
>>>>>>> own
>>>>>>> plugin upon which runtimes.core and runtime.ui can depend.
>>>>>>>
>>>>>>> The main negative of pushing stacks into base/runtimes, in my 
>>>>>>> opinion,
>>>>>>> is that there are a significant number of libraries required. 
>>>>>>> It's not
>>>>>>> too much, by far, but it is about 7 jars totalling about 1 
>>>>>>> megabyte.
>>>>>>> Whether these jars belong in base/runtimes is debatable, and 
>>>>>>> currently
>>>>>>> we do not have a "3rd-party dependencies" section in base where we
>>>>>>> organize common dependencies and versions together so that each 
>>>>>>> plugin
>>>>>>> doesn't need to bundle their own 3rd party libraries. I admit, 
>>>>>>> this is
>>>>>>> a debate for another time, but, I just wanted to point out that
>>>>>>> pushing the stacks logic down into runtimes would be another 
>>>>>>> example
>>>>>>> of this issue.
>>>>>>>
>>>>>>> Even still, I would argue that we should push stacks into its own
>>>>>>> small plugin below runtimes, deprecate the "downloadRuntimes"
>>>>>>> extension point, and the online downloadRuntime.xml (wherever 
>>>>>>> the file
>>>>>>> is, I forget).
>>>>>>>
>>>>>>> However, once we do that, there are many more questions. The 
>>>>>>> first is,
>>>>>>> who's job is it to provide the yaml file from which stacks are
>>>>>>> generated?
>>>>>>>
>>>>>>> Currently there is only one yaml file, and it is referenced 
>>>>>>> directly
>>>>>>> via a github url. Aside from how (IMO) this is fairly crazy in 
>>>>>>> itself,
>>>>>>> it causes another problem. The stacks client jar *can* cache the 
>>>>>>> yaml
>>>>>>> file and only update if the timestamp has changed, however, when
>>>>>>> checking the timestamp on a github file, there isn't one...
>>>>>>>
>>>>>>> This would seem to imply we should take control of the yaml file
>>>>>>> ourselves and put it NOT in github but rather in a release-specific
>>>>>>> online-accessible folder, ex: jbt4.1/stacks.yaml, 
>>>>>>> jbt5.0/stacks.yaml,
>>>>>>> etc.
>>>>>>>
>>>>>>> The problem with this is that we are then taking control away 
>>>>>>> from the
>>>>>>> jdf team, and once we take the file away, it is our job to keep it
>>>>>>> updated and in synch. This may cause errors if we are not very
>>>>>>> careful.
>>>>>>>
>>>>>>> Assuming we do this, though, the next question is, do we add 
>>>>>>> seam and
>>>>>>> esb runtimes to this yaml file, which currently only provides
>>>>>>> application servers? Remember, the purpose of moving stacks down 
>>>>>>> would
>>>>>>> be to deprecate the downloadRuntime extension point, therefore any
>>>>>>> replacement would need to do everything downloadRuntime does, which
>>>>>>> includes providing seam and esb runtimes for download.
>>>>>>>
>>>>>>> Let's assume (for now) that we simply add lines to the yaml to 
>>>>>>> allow
>>>>>>> it to provide seam and esb runtimes. We may come back to this point
>>>>>>> later, but for now, assume we do that.
>>>>>>>
>>>>>>>
>>>>>>> Then which plugin will provide the url to our copied yaml? Who's
>>>>>>> responsibility is it to point to this yaml file? Let's look at our
>>>>>>> options:
>>>>>>>
>>>>>>> 1) The runtimes plugin references the yaml
>>>>>>> 2) The central plugin references the yaml
>>>>>>>
>>>>>>> Both of these fail after thinking about it. How?
>>>>>>>
>>>>>>> 1) If the runtimes plugin references the yaml, then the download
>>>>>>> runtimes dialog will list things (like seam) which may not be 
>>>>>>> present
>>>>>>> in the installation. Imagine an installation with only base and 
>>>>>>> server
>>>>>>> plugins installed, and so no seam or esb. A user clicking 'download
>>>>>>> runtimes' will see esb and seam downloads, but the plugins which 
>>>>>>> are
>>>>>>> prepared to handle those runtimes after the download are not 
>>>>>>> present.
>>>>>>>
>>>>>>> 2) If central is in charge of providing this yaml, perhaps 
>>>>>>> through a
>>>>>>> new extension point to the base/runtimes/stacks plugin we add 
>>>>>>> there,
>>>>>>> then an installation including only plugins from base / server will
>>>>>>> have a BLANK download list. Users who install only ASTools will 
>>>>>>> not be
>>>>>>> able to download JBoss Application Servers.
>>>>>>>
>>>>>>> So both of these fail in their own way. The only solution as I can
>>>>>>> see, the only way it would work, would be to have multiple such 
>>>>>>> yaml
>>>>>>> files, one for astools, one for seam, one for esb, etc. Each of 
>>>>>>> these
>>>>>>> modules would provide their own yaml url to base/runtimes/stacks 
>>>>>>> via
>>>>>>> an extension point in base/runtimes/stacks, and let stacks fetch 
>>>>>>> each
>>>>>>> one and build a unified model.
>>>>>>>
>>>>>>> Problems:
>>>>>>>      a) multiple urls need to be loaded
>>>>>>>      b) multiple yaml files need to be kept up to date, instead 
>>>>>>> of just
>>>>>>> one. Multiply number of contributing plugins by number of major
>>>>>>> releases
>>>>>>>      c) Possibility of duplicates. Once you have multiple yaml 
>>>>>>> files
>>>>>>> generating models, it's possible some duplication leaks in. I'm 
>>>>>>> not so
>>>>>>> sure about this one, but Fred listed it as a concern.
>>>>>>>
>>>>>>> So, by my analysis, this is the only way I can imagine a 
>>>>>>> unification
>>>>>>> of these three models. I'll summarize the changes below, but it 
>>>>>>> does
>>>>>>> seem there would be a bit of work to do.
>>>>>>>
>>>>>>> Summary of changes:
>>>>>>>     1) Deprecate downloadRuntimes extension point
>>>>>>>     2) Create new plugin in runtimes module called "stacks"
>>>>>>>     3) Add extension point to 'stacks' plugin called stacksProvider
>>>>>>>     4) modify runtime.core and runtime.ui to use the model built in
>>>>>>> 'stacks'
>>>>>>>     5) Create a web-accessible location for jbt-release-relevent
>>>>>>> data on
>>>>>>> a per-module basis. For example,
>>>>>>> http://wherever/jbt/4.1.0/stacks/astools.yaml, etc.
>>>>>>>     6) Copy the current jdf yaml file to that location for 
>>>>>>> astools.yaml
>>>>>>>     7) Create a new yaml file which can build stacks for esb, 
>>>>>>> seam, etc
>>>>>>>     8) Ensure astools, esb, seam, etc, make use of the new
>>>>>>> stacksProvider
>>>>>>> extension point
>>>>>>>     9) Test the shit out of it
>>>>>>>
>>>>>>> There are other benefits to this approach. Currently there's no 
>>>>>>> really
>>>>>>> good mapping of downloadRuntimes id's to an app-server id. This is
>>>>>>> done in a hard-coded fashion in astools. This could instead be 
>>>>>>> added
>>>>>>> to the labels in the astools.yaml file if desired. It would allow
>>>>>>> dynamic addition or removal of any runtimes, though in the yaml
>>>>>>> syntax. It would minimize connections and re-downloads of the yaml
>>>>>>> files, since they'll actually have a timestamp now (as opposed 
>>>>>>> to in
>>>>>>> github, where they don't). And it could help clean up some other 
>>>>>>> areas
>>>>>>> that could benefit from a cleanup.
>>>>>>>
>>>>>>> I'd really like feedback on this issue from anyone who knows 
>>>>>>> anything
>>>>>>> about the topic, because I know for sure I'm lacking a bit in fully
>>>>>>> understanding the entire api. But I'd love at the least for 
>>>>>>> someone to
>>>>>>> tell me which of the logic here is obviously bad or if i'm wrong on
>>>>>>> any details.
>>>>>>>
>>>>>>> Thanks and look forward to the feedback
>>>>>>>
>>>>>>> - Rob Stryker
>>>>>>> I break things, and then put them back together.
>>>>>> _______________________________________________
>>>>>> jbosstools-dev mailing list
>>>>>> jbosstools-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>>>> _______________________________________________
>>>>> jbosstools-dev mailing list
>>>>> jbosstools-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>> _______________________________________________
>>> jbosstools-dev mailing list
>>> jbosstools-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jbosstools-dev/attachments/20130401/06c54a54/attachment-0001.html

More information about the jbosstools-dev mailing list