[jbosstools-dev] Regarding Stacks, Runtimes, and Remote Descriptors

Max Rydahl Andersen max.andersen at redhat.com
Tue Apr 2 07:39:33 EDT 2013 

Hi Rafael,

We also got runtimes like drools and seam which aren't exactly servers.

Also, can we have runtimes listed without having matching archetypes ?
i.e. if we would like to move all our existing info over to stacks.yml
instead of having it spread over we would need this runtime list without
having matching bom's etc.

/max

On Mon, Apr 01, 2013 at 09:25:52AM -0300, Rafael Benevides wrote:
>Hi!
>
>Since we can always add these informations as we need. I'm using a "on 
>demand"  approach, where someone or team who needs it, I ask to 
>provide a Pull Request with the Runtime/BOM/Archetype needed. Them I 
>test again and merge. Anyone can do the tests with 'mvn test'.
>Just this :)
>
>Since we haven't any other Runtimes than AS and EAP, I think we can 
>use the following ' runtime-type': JPP, SOAP, BRMS, GateIn, etc and 
>keep the same 'runtime-category: SERVER'.
>
>Thank you :)
>
>
>Em 01/04/13 05:39, Rob Stryker escreveu:
>>Hi Rafael:
>>
>>I'm sure I and the tools team will have more questions in the 
>>future, but, right now one question I have is whether it's planned 
>>to add ALL of our runtimes to the stacks? Things like jpp, GateIn, 
>>SOA-P, etc?
>>
>>So far it seems limited to AS and EAP.
>>
>>Thanks!
>>
>>- Rob Stryker
>>
>>On 03/27/2013 12:39 AM, Rafael Benevides wrote:
>>>Hi Fred and all JBDS team!
>>>
>>>The idea of "merge data from separate yaml files" doesn't worked 
>>>and we (JDF team) didn't put any more effort on it.
>>>
>>>The original idea is that we could join the information (appending 
>>>a new content to the original stacks.yaml) and use its YAML 
>>>anchors. To clarify what I'm saying, look the matrix.yaml file (https://github.com/jboss-jdf/jdf-stack/blob/1.0.0.Final/matrix.yaml) 
>>>it has some references to BOMs and archtypes that is defined on 
>>>original stacks.yaml.
>>>
>>>The main problem is that the new generarate yaml file is not 
>>>parsable via StacksClient (because it expects the original 1.0.0 
>>>format only) and a new "client" should be built to each new format 
>>>extension. This could make us loose the control since any data 
>>>change on stacks.uaml could impact all extensions that uses it. So 
>>>in fact: It was not a good idea to have this extensions on 
>>>separate files.
>>>
>>>The next "Stacks 1.0" planned evolution is 
>>>https://issues.jboss.org/browse/JDF-222 - Stacks should permit a 
>>>"Early Access" Runtimes - And because of the file format, this 
>>>"Early Acccess" will be a new "label" but I have afraid that 
>>>consumers must know that "allAvailableRuntimes" has 
>>>"Released/Final" and "Early Access" runtimes mixed. They should 
>>>query the labels to distinguish them.
>>>
>>>That's my 2 cents.
>>>
>>>
>>>Em 26/03/13 10:25, Fred Bricon escreveu:
>>>>Hey Rafael,
>>>>
>>>>what's the status on the "merge data from separate yaml files"? 
>>>>I know you worked on that at some point.
>>>>We'll most certainly need to combine JBoss Tools specific 
>>>>runtimes or infos with the existing JDF ones.
>>>>
>>>>Fred
>>>>
>>>>Le 02/03/2013 08:52, Rob Stryker a écrit :
>>>>>license, size, and disclaimer can all be added in the labels 
>>>>>section of
>>>>>the yaml.
>>>>>
>>>>>Max has already made it clear that having three things that do 
>>>>>the same
>>>>>thing is wasteful and confusing, so the question isn't *if* we will
>>>>>unify them, but rather *how*.
>>>>>
>>>>>On 03/01/2013 09:29 PM, Snjezana Peco wrote:
>>>>>>I think we need to keep all of those options (since they already
>>>>>>exist). Each of them has its advantages/disadvantages.
>>>>>>I agree with the proposed changes and emphasize that stacks 
>>>>>>don't have
>>>>>>the following properties:
>>>>>>
>>>>>>- license
>>>>>>- size
>>>>>>- requireSso (see 
>>>>>>https://github.com/jbosstools/jbosstools-base/pull/50)
>>>>>>- disclaimer
>>>>>>
>>>>>>Snjeza
>>>>>>
>>>>>>On 3/1/2013 10:22 AM, Rob Stryker wrote:
>>>>>>>I would like to hear some feedback here from Max and Snjezana.
>>>>>>>
>>>>>>>On 02/28/2013 08:05 PM, Fred Bricon wrote:
>>>>>>>>I  mostly agree with the changes you described. Here's my 0.02€ :
>>>>>>>>- I strongly believe runtimes should be split into different stacks
>>>>>>>>descriptors, but I don't like the idea of having to 
>>>>>>>>maintain a fork of
>>>>>>>>the JDF one. We should only add extensions (old AS'es, 
>>>>>>>>seam, ESB ...).
>>>>>>>>- I believe the merge all stacks descriptors in one 
>>>>>>>>metamodel should be
>>>>>>>>done in stacks client. I know Raphael kinda started 
>>>>>>>>working on that a
>>>>>>>>few months back, he probably can give us his insight.
>>>>>>>>- runtimes in stacks could list their managing JBT plugins in the
>>>>>>>>labels property : i.e if ESB runtime can be downloaded, but no ESB
>>>>>>>>plugin is installed, the we'd be able to discover both 
>>>>>>>>the runtime AND
>>>>>>>>its associated plugin
>>>>>>>>
>>>>>>>>Fred
>>>>>>>>
>>>>>>>>Le mercredi 27 février 2013 18:22:35, Rob Stryker a écrit :
>>>>>>>>>Regarding Stacks, Runtimes, and Remote Descriptors
>>>>>>>>>
>>>>>>>>>Hi All:
>>>>>>>>>
>>>>>>>>>This email is to try to begin discussion on some 
>>>>>>>>>recent duplication of
>>>>>>>>>code and responsibilities, which should probably be fixed before
>>>>>>>>>things get too comfortable.  I'm speaking specifically 
>>>>>>>>>about the role
>>>>>>>>>of discovering runtimes to download, where that's done, how that's
>>>>>>>>>done, and which responsibility belongs to who. Forgive 
>>>>>>>>>me if the email
>>>>>>>>>is long, as I am trying to be thorough.
>>>>>>>>>
>>>>>>>>>Currently, there are three places from which runtimes 
>>>>>>>>>to download may
>>>>>>>>>be discovered.
>>>>>>>>>
>>>>>>>>>1) base/runtimes has an extension point named 
>>>>>>>>>downloadRuntimes, which
>>>>>>>>>is used by AS Tools and Seam Tools (and perhaps others).
>>>>>>>>>
>>>>>>>>>2) a remote descriptor file which acts as a second arm 
>>>>>>>>>of 1) and is
>>>>>>>>>basically an xml form of 1) used to dynamically add 
>>>>>>>>>new runtimes as
>>>>>>>>>they become available
>>>>>>>>>
>>>>>>>>>3) The new Stacks methodology, currently stored in
>>>>>>>>>jbosstools-central/maven/plugins/org.jboss.tools.maven.project.examples
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>We should begin unifying these three locations into 
>>>>>>>>>one, but the goal
>>>>>>>>>is to do it correctly. So, I would first like to list 
>>>>>>>>>the benefits of
>>>>>>>>>each.
>>>>>>>>>
>>>>>>>>>a) downloadable runtimes provided through the 
>>>>>>>>>extension point cannot
>>>>>>>>>be removed without a maintenance or major release of 
>>>>>>>>>some type, and
>>>>>>>>>for this reason are semi-permanent
>>>>>>>>>
>>>>>>>>>b) downloadable runtimes available via the remote 
>>>>>>>>>descriptor file may
>>>>>>>>>be added OR removed at will. This provides flexibility and
>>>>>>>>>post-release updates are easy.
>>>>>>>>>
>>>>>>>>>c) The new stacks section has a more robust model 
>>>>>>>>>capable of providing
>>>>>>>>>more information than the downloadable runtimes does. However, the
>>>>>>>>>plugin requires several libraries and is currently placed in the
>>>>>>>>>jboss-central module, where others may not make use of it.
>>>>>>>>>
>>>>>>>>>d) the Stacks yaml file does not provide a place to 
>>>>>>>>>access the file
>>>>>>>>>size for the download, however it does provide a 'labels' section,
>>>>>>>>>which seems extendable to add whatever properties you 
>>>>>>>>>may want to add.
>>>>>>>>>
>>>>>>>>>At first glance, it seems that Stacks is the superior 
>>>>>>>>>framework. It is
>>>>>>>>>extensible, it can have unlimited labels (aka 
>>>>>>>>>properties) if desired,
>>>>>>>>>and it already provides more information which is 
>>>>>>>>>usable to others who
>>>>>>>>>may want it. To make use of stacks inside Runtimes, however, we'd
>>>>>>>>>either need to:
>>>>>>>>>      a) Expand the API in runtimes to allow other 
>>>>>>>>>plugins (like in
>>>>>>>>>central) to provide downloadable runtimes,  or,
>>>>>>>>>      b) Push 'stacks' out of central and down into 
>>>>>>>>>runtimes, as its
>>>>>>>>>own
>>>>>>>>>plugin upon which runtimes.core and runtime.ui can depend.
>>>>>>>>>
>>>>>>>>>The main negative of pushing stacks into 
>>>>>>>>>base/runtimes, in my opinion,
>>>>>>>>>is that there are a significant number of libraries 
>>>>>>>>>required. It's not
>>>>>>>>>too much, by far, but it is about 7 jars totalling 
>>>>>>>>>about 1 megabyte.
>>>>>>>>>Whether these jars belong in base/runtimes is 
>>>>>>>>>debatable, and currently
>>>>>>>>>we do not have a "3rd-party dependencies" section in base where we
>>>>>>>>>organize common dependencies and versions together so 
>>>>>>>>>that each plugin
>>>>>>>>>doesn't need to bundle their own 3rd party libraries. 
>>>>>>>>>I admit, this is
>>>>>>>>>a debate for another time, but, I just wanted to point out that
>>>>>>>>>pushing the stacks logic down into runtimes would be 
>>>>>>>>>another example
>>>>>>>>>of this issue.
>>>>>>>>>
>>>>>>>>>Even still, I would argue that we should push stacks into its own
>>>>>>>>>small plugin below runtimes, deprecate the "downloadRuntimes"
>>>>>>>>>extension point, and the online downloadRuntime.xml 
>>>>>>>>>(wherever the file
>>>>>>>>>is, I forget).
>>>>>>>>>
>>>>>>>>>However, once we do that, there are many more 
>>>>>>>>>questions. The first is,
>>>>>>>>>who's job is it to provide the yaml file from which stacks are
>>>>>>>>>generated?
>>>>>>>>>
>>>>>>>>>Currently there is only one yaml file, and it is 
>>>>>>>>>referenced directly
>>>>>>>>>via a github url. Aside from how (IMO) this is fairly 
>>>>>>>>>crazy in itself,
>>>>>>>>>it causes another problem. The stacks client jar *can* 
>>>>>>>>>cache the yaml
>>>>>>>>>file and only update if the timestamp has changed, however, when
>>>>>>>>>checking the timestamp on a github file, there isn't one...
>>>>>>>>>
>>>>>>>>>This would seem to imply we should take control of the yaml file
>>>>>>>>>ourselves and put it NOT in github but rather in a 
>>>>>>>>>release-specific
>>>>>>>>>online-accessible folder, ex: jbt4.1/stacks.yaml, 
>>>>>>>>>jbt5.0/stacks.yaml,
>>>>>>>>>etc.
>>>>>>>>>
>>>>>>>>>The problem with this is that we are then taking 
>>>>>>>>>control away from the
>>>>>>>>>jdf team, and once we take the file away, it is our job to keep it
>>>>>>>>>updated and in synch. This may cause errors if we are not very
>>>>>>>>>careful.
>>>>>>>>>
>>>>>>>>>Assuming we do this, though, the next question is, do 
>>>>>>>>>we add seam and
>>>>>>>>>esb runtimes to this yaml file, which currently only provides
>>>>>>>>>application servers? Remember, the purpose of moving 
>>>>>>>>>stacks down would
>>>>>>>>>be to deprecate the downloadRuntime extension point, therefore any
>>>>>>>>>replacement would need to do everything 
>>>>>>>>>downloadRuntime does, which
>>>>>>>>>includes providing seam and esb runtimes for download.
>>>>>>>>>
>>>>>>>>>Let's assume (for now) that we simply add lines to the 
>>>>>>>>>yaml to allow
>>>>>>>>>it to provide seam and esb runtimes. We may come back 
>>>>>>>>>to this point
>>>>>>>>>later, but for now, assume we do that.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>Then which plugin will provide the url to our copied yaml? Who's
>>>>>>>>>responsibility is it to point to this yaml file? Let's look at our
>>>>>>>>>options:
>>>>>>>>>
>>>>>>>>>1) The runtimes plugin references the yaml
>>>>>>>>>2) The central plugin references the yaml
>>>>>>>>>
>>>>>>>>>Both of these fail after thinking about it. How?
>>>>>>>>>
>>>>>>>>>1) If the runtimes plugin references the yaml, then the download
>>>>>>>>>runtimes dialog will list things (like seam) which may 
>>>>>>>>>not be present
>>>>>>>>>in the installation. Imagine an installation with only 
>>>>>>>>>base and server
>>>>>>>>>plugins installed, and so no seam or esb. A user 
>>>>>>>>>clicking 'download
>>>>>>>>>runtimes' will see esb and seam downloads, but the 
>>>>>>>>>plugins which are
>>>>>>>>>prepared to handle those runtimes after the download 
>>>>>>>>>are not present.
>>>>>>>>>
>>>>>>>>>2) If central is in charge of providing this yaml, 
>>>>>>>>>perhaps through a
>>>>>>>>>new extension point to the base/runtimes/stacks plugin 
>>>>>>>>>we add there,
>>>>>>>>>then an installation including only plugins from base 
>>>>>>>>>/ server will
>>>>>>>>>have a BLANK download list. Users who install only 
>>>>>>>>>ASTools will not be
>>>>>>>>>able to download JBoss Application Servers.
>>>>>>>>>
>>>>>>>>>So both of these fail in their own way. The only solution as I can
>>>>>>>>>see, the only way it would work, would be to have 
>>>>>>>>>multiple such yaml
>>>>>>>>>files, one for astools, one for seam, one for esb, 
>>>>>>>>>etc. Each of these
>>>>>>>>>modules would provide their own yaml url to 
>>>>>>>>>base/runtimes/stacks via
>>>>>>>>>an extension point in base/runtimes/stacks, and let 
>>>>>>>>>stacks fetch each
>>>>>>>>>one and build a unified model.
>>>>>>>>>
>>>>>>>>>Problems:
>>>>>>>>>     a) multiple urls need to be loaded
>>>>>>>>>     b) multiple yaml files need to be kept up to 
>>>>>>>>>date, instead of just
>>>>>>>>>one. Multiply number of contributing plugins by number of major
>>>>>>>>>releases
>>>>>>>>>     c) Possibility of duplicates. Once you have 
>>>>>>>>>multiple yaml files
>>>>>>>>>generating models, it's possible some duplication 
>>>>>>>>>leaks in. I'm not so
>>>>>>>>>sure about this one, but Fred listed it as a concern.
>>>>>>>>>
>>>>>>>>>So, by my analysis, this is the only way I can imagine 
>>>>>>>>>a unification
>>>>>>>>>of these three models. I'll summarize the changes 
>>>>>>>>>below, but it does
>>>>>>>>>seem there would be a bit of work to do.
>>>>>>>>>
>>>>>>>>>Summary of changes:
>>>>>>>>>    1) Deprecate downloadRuntimes extension point
>>>>>>>>>    2) Create new plugin in runtimes module called "stacks"
>>>>>>>>>    3) Add extension point to 'stacks' plugin called 
>>>>>>>>>stacksProvider
>>>>>>>>>    4) modify runtime.core and runtime.ui to use the 
>>>>>>>>>model built in
>>>>>>>>>'stacks'
>>>>>>>>>    5) Create a web-accessible location for jbt-release-relevent
>>>>>>>>>data on
>>>>>>>>>a per-module basis. For example,
>>>>>>>>>http://wherever/jbt/4.1.0/stacks/astools.yaml, etc.
>>>>>>>>>    6) Copy the current jdf yaml file to that location 
>>>>>>>>>for astools.yaml
>>>>>>>>>    7) Create a new yaml file which can build stacks 
>>>>>>>>>for esb, seam, etc
>>>>>>>>>    8) Ensure astools, esb, seam, etc, make use of the new
>>>>>>>>>stacksProvider
>>>>>>>>>extension point
>>>>>>>>>    9) Test the shit out of it
>>>>>>>>>
>>>>>>>>>There are other benefits to this approach. Currently 
>>>>>>>>>there's no really
>>>>>>>>>good mapping of downloadRuntimes id's to an app-server id. This is
>>>>>>>>>done in a hard-coded fashion in astools. This could 
>>>>>>>>>instead be added
>>>>>>>>>to the labels in the astools.yaml file if desired. It would allow
>>>>>>>>>dynamic addition or removal of any runtimes, though in the yaml
>>>>>>>>>syntax. It would minimize connections and re-downloads of the yaml
>>>>>>>>>files, since they'll actually have a timestamp now (as 
>>>>>>>>>opposed to in
>>>>>>>>>github, where they don't). And it could help clean up 
>>>>>>>>>some other areas
>>>>>>>>>that could benefit from a cleanup.
>>>>>>>>>
>>>>>>>>>I'd really like feedback on this issue from anyone who 
>>>>>>>>>knows anything
>>>>>>>>>about the topic, because I know for sure I'm lacking a 
>>>>>>>>>bit in fully
>>>>>>>>>understanding the entire api. But I'd love at the 
>>>>>>>>>least for someone to
>>>>>>>>>tell me which of the logic here is obviously bad or if 
>>>>>>>>>i'm wrong on
>>>>>>>>>any details.
>>>>>>>>>
>>>>>>>>>Thanks and look forward to the feedback
>>>>>>>>>
>>>>>>>>>- Rob Stryker
>>>>>>>>>I break things, and then put them back together.
>>>>>>>>_______________________________________________
>>>>>>>>jbosstools-dev mailing list
>>>>>>>>jbosstools-dev at lists.jboss.org
>>>>>>>>https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>>>>>>_______________________________________________
>>>>>>>jbosstools-dev mailing list
>>>>>>>jbosstools-dev at lists.jboss.org
>>>>>>>https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>>>>_______________________________________________
>>>>>jbosstools-dev mailing list
>>>>>jbosstools-dev at lists.jboss.org
>>>>>https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>>>
>>>
>>
>

>_______________________________________________
>jbosstools-dev mailing list
>jbosstools-dev at lists.jboss.org
>https://lists.jboss.org/mailman/listinfo/jbosstools-dev

More information about the jbosstools-dev mailing list