[jbosstools-dev] Regarding Stacks, Runtimes, and Remote Descriptors

Rafael Benevides benevides at redhat.com
Mon Apr 1 08:25:52 EDT 2013 

Hi!

Since we can always add these informations as we need. I'm using a "on 
demand"  approach, where someone or team who needs it, I ask to provide 
a Pull Request with the Runtime/BOM/Archetype needed. Them I test again 
and merge. Anyone can do the tests with 'mvn test'.
Just this :)

Since we haven't any other Runtimes than AS and EAP, I think we can use 
the following ' runtime-type': JPP, SOAP, BRMS, GateIn, etc and keep the 
same 'runtime-category: SERVER'.

Thank you :)


Em 01/04/13 05:39, Rob Stryker escreveu:
> Hi Rafael:
>
> I'm sure I and the tools team will have more questions in the future, 
> but, right now one question I have is whether it's planned to add ALL 
> of our runtimes to the stacks? Things like jpp, GateIn, SOA-P, etc?
>
> So far it seems limited to AS and EAP.
>
> Thanks!
>
> - Rob Stryker
>
> On 03/27/2013 12:39 AM, Rafael Benevides wrote:
>> Hi Fred and all JBDS team!
>>
>> The idea of "merge data from separate yaml files" doesn't worked and 
>> we (JDF team) didn't put any more effort on it.
>>
>> The original idea is that we could join the information (appending a 
>> new content to the original stacks.yaml) and use its YAML anchors. To 
>> clarify what I'm saying, look the matrix.yaml file 
>> (https://github.com/jboss-jdf/jdf-stack/blob/1.0.0.Final/matrix.yaml) 
>> it has some references to BOMs and archtypes that is defined on 
>> original stacks.yaml.
>>
>> The main problem is that the new generarate yaml file is not parsable 
>> via StacksClient (because it expects the original 1.0.0 format only) 
>> and a new "client" should be built to each new format extension. This 
>> could make us loose the control since any data change on stacks.uaml 
>> could impact all extensions that uses it. So in fact: It was not a 
>> good idea to have this extensions on separate files.
>>
>> The next "Stacks 1.0" planned evolution is 
>> https://issues.jboss.org/browse/JDF-222 - Stacks should permit a 
>> "Early Access" Runtimes - And because of the file format, this "Early 
>> Acccess" will be a new "label" but I have afraid that consumers must 
>> know that "allAvailableRuntimes" has "Released/Final" and "Early 
>> Access" runtimes mixed. They should query the labels to distinguish them.
>>
>> That's my 2 cents.
>>
>>
>> Em 26/03/13 10:25, Fred Bricon escreveu:
>>> Hey Rafael,
>>>
>>> what's the status on the "merge data from separate yaml files"? I 
>>> know you worked on that at some point.
>>> We'll most certainly need to combine JBoss Tools specific runtimes 
>>> or infos with the existing JDF ones.
>>>
>>> Fred
>>>
>>> Le 02/03/2013 08:52, Rob Stryker a écrit :
>>>> license, size, and disclaimer can all be added in the labels 
>>>> section of
>>>> the yaml.
>>>>
>>>> Max has already made it clear that having three things that do the 
>>>> same
>>>> thing is wasteful and confusing, so the question isn't *if* we will
>>>> unify them, but rather *how*.
>>>>
>>>> On 03/01/2013 09:29 PM, Snjezana Peco wrote:
>>>>> I think we need to keep all of those options (since they already
>>>>> exist). Each of them has its advantages/disadvantages.
>>>>> I agree with the proposed changes and emphasize that stacks don't 
>>>>> have
>>>>> the following properties:
>>>>>
>>>>> - license
>>>>> - size
>>>>> - requireSso (see 
>>>>> https://github.com/jbosstools/jbosstools-base/pull/50)
>>>>> - disclaimer
>>>>>
>>>>> Snjeza
>>>>>
>>>>> On 3/1/2013 10:22 AM, Rob Stryker wrote:
>>>>>> I would like to hear some feedback here from Max and Snjezana.
>>>>>>
>>>>>> On 02/28/2013 08:05 PM, Fred Bricon wrote:
>>>>>>> I  mostly agree with the changes you described. Here's my 0.02€ :
>>>>>>> - I strongly believe runtimes should be split into different stacks
>>>>>>> descriptors, but I don't like the idea of having to maintain a 
>>>>>>> fork of
>>>>>>> the JDF one. We should only add extensions (old AS'es, seam, ESB 
>>>>>>> ...).
>>>>>>> - I believe the merge all stacks descriptors in one metamodel 
>>>>>>> should be
>>>>>>> done in stacks client. I know Raphael kinda started working on 
>>>>>>> that a
>>>>>>> few months back, he probably can give us his insight.
>>>>>>> - runtimes in stacks could list their managing JBT plugins in the
>>>>>>> labels property : i.e if ESB runtime can be downloaded, but no ESB
>>>>>>> plugin is installed, the we'd be able to discover both the 
>>>>>>> runtime AND
>>>>>>> its associated plugin
>>>>>>>
>>>>>>> Fred
>>>>>>>
>>>>>>> Le mercredi 27 février 2013 18:22:35, Rob Stryker a écrit :
>>>>>>>> Regarding Stacks, Runtimes, and Remote Descriptors
>>>>>>>>
>>>>>>>> Hi All:
>>>>>>>>
>>>>>>>> This email is to try to begin discussion on some recent 
>>>>>>>> duplication of
>>>>>>>> code and responsibilities, which should probably be fixed before
>>>>>>>> things get too comfortable.  I'm speaking specifically about 
>>>>>>>> the role
>>>>>>>> of discovering runtimes to download, where that's done, how that's
>>>>>>>> done, and which responsibility belongs to who. Forgive me if 
>>>>>>>> the email
>>>>>>>> is long, as I am trying to be thorough.
>>>>>>>>
>>>>>>>> Currently, there are three places from which runtimes to 
>>>>>>>> download may
>>>>>>>> be discovered.
>>>>>>>>
>>>>>>>> 1) base/runtimes has an extension point named downloadRuntimes, 
>>>>>>>> which
>>>>>>>> is used by AS Tools and Seam Tools (and perhaps others).
>>>>>>>>
>>>>>>>> 2) a remote descriptor file which acts as a second arm of 1) 
>>>>>>>> and is
>>>>>>>> basically an xml form of 1) used to dynamically add new 
>>>>>>>> runtimes as
>>>>>>>> they become available
>>>>>>>>
>>>>>>>> 3) The new Stacks methodology, currently stored in
>>>>>>>> jbosstools-central/maven/plugins/org.jboss.tools.maven.project.examples 
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> We should begin unifying these three locations into one, but 
>>>>>>>> the goal
>>>>>>>> is to do it correctly. So, I would first like to list the 
>>>>>>>> benefits of
>>>>>>>> each.
>>>>>>>>
>>>>>>>> a) downloadable runtimes provided through the extension point 
>>>>>>>> cannot
>>>>>>>> be removed without a maintenance or major release of some type, 
>>>>>>>> and
>>>>>>>> for this reason are semi-permanent
>>>>>>>>
>>>>>>>> b) downloadable runtimes available via the remote descriptor 
>>>>>>>> file may
>>>>>>>> be added OR removed at will. This provides flexibility and
>>>>>>>> post-release updates are easy.
>>>>>>>>
>>>>>>>> c) The new stacks section has a more robust model capable of 
>>>>>>>> providing
>>>>>>>> more information than the downloadable runtimes does. However, the
>>>>>>>> plugin requires several libraries and is currently placed in the
>>>>>>>> jboss-central module, where others may not make use of it.
>>>>>>>>
>>>>>>>> d) the Stacks yaml file does not provide a place to access the 
>>>>>>>> file
>>>>>>>> size for the download, however it does provide a 'labels' section,
>>>>>>>> which seems extendable to add whatever properties you may want 
>>>>>>>> to add.
>>>>>>>>
>>>>>>>> At first glance, it seems that Stacks is the superior 
>>>>>>>> framework. It is
>>>>>>>> extensible, it can have unlimited labels (aka properties) if 
>>>>>>>> desired,
>>>>>>>> and it already provides more information which is usable to 
>>>>>>>> others who
>>>>>>>> may want it. To make use of stacks inside Runtimes, however, we'd
>>>>>>>> either need to:
>>>>>>>>       a) Expand the API in runtimes to allow other plugins 
>>>>>>>> (like in
>>>>>>>> central) to provide downloadable runtimes,  or,
>>>>>>>>       b) Push 'stacks' out of central and down into runtimes, 
>>>>>>>> as its
>>>>>>>> own
>>>>>>>> plugin upon which runtimes.core and runtime.ui can depend.
>>>>>>>>
>>>>>>>> The main negative of pushing stacks into base/runtimes, in my 
>>>>>>>> opinion,
>>>>>>>> is that there are a significant number of libraries required. 
>>>>>>>> It's not
>>>>>>>> too much, by far, but it is about 7 jars totalling about 1 
>>>>>>>> megabyte.
>>>>>>>> Whether these jars belong in base/runtimes is debatable, and 
>>>>>>>> currently
>>>>>>>> we do not have a "3rd-party dependencies" section in base where we
>>>>>>>> organize common dependencies and versions together so that each 
>>>>>>>> plugin
>>>>>>>> doesn't need to bundle their own 3rd party libraries. I admit, 
>>>>>>>> this is
>>>>>>>> a debate for another time, but, I just wanted to point out that
>>>>>>>> pushing the stacks logic down into runtimes would be another 
>>>>>>>> example
>>>>>>>> of this issue.
>>>>>>>>
>>>>>>>> Even still, I would argue that we should push stacks into its own
>>>>>>>> small plugin below runtimes, deprecate the "downloadRuntimes"
>>>>>>>> extension point, and the online downloadRuntime.xml (wherever 
>>>>>>>> the file
>>>>>>>> is, I forget).
>>>>>>>>
>>>>>>>> However, once we do that, there are many more questions. The 
>>>>>>>> first is,
>>>>>>>> who's job is it to provide the yaml file from which stacks are
>>>>>>>> generated?
>>>>>>>>
>>>>>>>> Currently there is only one yaml file, and it is referenced 
>>>>>>>> directly
>>>>>>>> via a github url. Aside from how (IMO) this is fairly crazy in 
>>>>>>>> itself,
>>>>>>>> it causes another problem. The stacks client jar *can* cache 
>>>>>>>> the yaml
>>>>>>>> file and only update if the timestamp has changed, however, when
>>>>>>>> checking the timestamp on a github file, there isn't one...
>>>>>>>>
>>>>>>>> This would seem to imply we should take control of the yaml file
>>>>>>>> ourselves and put it NOT in github but rather in a 
>>>>>>>> release-specific
>>>>>>>> online-accessible folder, ex: jbt4.1/stacks.yaml, 
>>>>>>>> jbt5.0/stacks.yaml,
>>>>>>>> etc.
>>>>>>>>
>>>>>>>> The problem with this is that we are then taking control away 
>>>>>>>> from the
>>>>>>>> jdf team, and once we take the file away, it is our job to keep it
>>>>>>>> updated and in synch. This may cause errors if we are not very
>>>>>>>> careful.
>>>>>>>>
>>>>>>>> Assuming we do this, though, the next question is, do we add 
>>>>>>>> seam and
>>>>>>>> esb runtimes to this yaml file, which currently only provides
>>>>>>>> application servers? Remember, the purpose of moving stacks 
>>>>>>>> down would
>>>>>>>> be to deprecate the downloadRuntime extension point, therefore any
>>>>>>>> replacement would need to do everything downloadRuntime does, 
>>>>>>>> which
>>>>>>>> includes providing seam and esb runtimes for download.
>>>>>>>>
>>>>>>>> Let's assume (for now) that we simply add lines to the yaml to 
>>>>>>>> allow
>>>>>>>> it to provide seam and esb runtimes. We may come back to this 
>>>>>>>> point
>>>>>>>> later, but for now, assume we do that.
>>>>>>>>
>>>>>>>>
>>>>>>>> Then which plugin will provide the url to our copied yaml? Who's
>>>>>>>> responsibility is it to point to this yaml file? Let's look at our
>>>>>>>> options:
>>>>>>>>
>>>>>>>> 1) The runtimes plugin references the yaml
>>>>>>>> 2) The central plugin references the yaml
>>>>>>>>
>>>>>>>> Both of these fail after thinking about it. How?
>>>>>>>>
>>>>>>>> 1) If the runtimes plugin references the yaml, then the download
>>>>>>>> runtimes dialog will list things (like seam) which may not be 
>>>>>>>> present
>>>>>>>> in the installation. Imagine an installation with only base and 
>>>>>>>> server
>>>>>>>> plugins installed, and so no seam or esb. A user clicking 
>>>>>>>> 'download
>>>>>>>> runtimes' will see esb and seam downloads, but the plugins 
>>>>>>>> which are
>>>>>>>> prepared to handle those runtimes after the download are not 
>>>>>>>> present.
>>>>>>>>
>>>>>>>> 2) If central is in charge of providing this yaml, perhaps 
>>>>>>>> through a
>>>>>>>> new extension point to the base/runtimes/stacks plugin we add 
>>>>>>>> there,
>>>>>>>> then an installation including only plugins from base / server 
>>>>>>>> will
>>>>>>>> have a BLANK download list. Users who install only ASTools will 
>>>>>>>> not be
>>>>>>>> able to download JBoss Application Servers.
>>>>>>>>
>>>>>>>> So both of these fail in their own way. The only solution as I can
>>>>>>>> see, the only way it would work, would be to have multiple such 
>>>>>>>> yaml
>>>>>>>> files, one for astools, one for seam, one for esb, etc. Each of 
>>>>>>>> these
>>>>>>>> modules would provide their own yaml url to 
>>>>>>>> base/runtimes/stacks via
>>>>>>>> an extension point in base/runtimes/stacks, and let stacks 
>>>>>>>> fetch each
>>>>>>>> one and build a unified model.
>>>>>>>>
>>>>>>>> Problems:
>>>>>>>>      a) multiple urls need to be loaded
>>>>>>>>      b) multiple yaml files need to be kept up to date, instead 
>>>>>>>> of just
>>>>>>>> one. Multiply number of contributing plugins by number of major
>>>>>>>> releases
>>>>>>>>      c) Possibility of duplicates. Once you have multiple yaml 
>>>>>>>> files
>>>>>>>> generating models, it's possible some duplication leaks in. I'm 
>>>>>>>> not so
>>>>>>>> sure about this one, but Fred listed it as a concern.
>>>>>>>>
>>>>>>>> So, by my analysis, this is the only way I can imagine a 
>>>>>>>> unification
>>>>>>>> of these three models. I'll summarize the changes below, but it 
>>>>>>>> does
>>>>>>>> seem there would be a bit of work to do.
>>>>>>>>
>>>>>>>> Summary of changes:
>>>>>>>>     1) Deprecate downloadRuntimes extension point
>>>>>>>>     2) Create new plugin in runtimes module called "stacks"
>>>>>>>>     3) Add extension point to 'stacks' plugin called 
>>>>>>>> stacksProvider
>>>>>>>>     4) modify runtime.core and runtime.ui to use the model 
>>>>>>>> built in
>>>>>>>> 'stacks'
>>>>>>>>     5) Create a web-accessible location for jbt-release-relevent
>>>>>>>> data on
>>>>>>>> a per-module basis. For example,
>>>>>>>> http://wherever/jbt/4.1.0/stacks/astools.yaml, etc.
>>>>>>>>     6) Copy the current jdf yaml file to that location for 
>>>>>>>> astools.yaml
>>>>>>>>     7) Create a new yaml file which can build stacks for esb, 
>>>>>>>> seam, etc
>>>>>>>>     8) Ensure astools, esb, seam, etc, make use of the new
>>>>>>>> stacksProvider
>>>>>>>> extension point
>>>>>>>>     9) Test the shit out of it
>>>>>>>>
>>>>>>>> There are other benefits to this approach. Currently there's no 
>>>>>>>> really
>>>>>>>> good mapping of downloadRuntimes id's to an app-server id. This is
>>>>>>>> done in a hard-coded fashion in astools. This could instead be 
>>>>>>>> added
>>>>>>>> to the labels in the astools.yaml file if desired. It would allow
>>>>>>>> dynamic addition or removal of any runtimes, though in the yaml
>>>>>>>> syntax. It would minimize connections and re-downloads of the yaml
>>>>>>>> files, since they'll actually have a timestamp now (as opposed 
>>>>>>>> to in
>>>>>>>> github, where they don't). And it could help clean up some 
>>>>>>>> other areas
>>>>>>>> that could benefit from a cleanup.
>>>>>>>>
>>>>>>>> I'd really like feedback on this issue from anyone who knows 
>>>>>>>> anything
>>>>>>>> about the topic, because I know for sure I'm lacking a bit in 
>>>>>>>> fully
>>>>>>>> understanding the entire api. But I'd love at the least for 
>>>>>>>> someone to
>>>>>>>> tell me which of the logic here is obviously bad or if i'm 
>>>>>>>> wrong on
>>>>>>>> any details.
>>>>>>>>
>>>>>>>> Thanks and look forward to the feedback
>>>>>>>>
>>>>>>>> - Rob Stryker
>>>>>>>> I break things, and then put them back together.
>>>>>>> _______________________________________________
>>>>>>> jbosstools-dev mailing list
>>>>>>> jbosstools-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>>>>> _______________________________________________
>>>>>> jbosstools-dev mailing list
>>>>>> jbosstools-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>>> _______________________________________________
>>>> jbosstools-dev mailing list
>>>> jbosstools-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jbosstools-dev/attachments/20130401/7cfbc1ee/attachment-0001.html

More information about the jbosstools-dev mailing list