In looking at the JBT 4.1.1.Alpha2 build [1], I see that some features have not yet changed since JBT 4.1.0.Final [2]. This is fine for a project like GWT or Freemarker, where the entire project is versioned the same way and we can simply include the previous .Final release, but for a subcomponent of a project, like the runtime component in jbosstools-base [3], this presents a question. [1] http://download.jboss.org/jbosstools/updates/nightly/core/4.1.kepler/ [2] http://download.jboss.org/jbosstools/updates/stable/kepler/ [3] http://download.jboss.org/jbosstools/builds/staging/jbosstools-base_41/al... Currently, the base build [3] is producing this plugin: org.jboss.tools.runtime.core 2.1.0.Alpha2-v20130927-0137-B114 But the JBT 4.1.1.Alpha2 build [1] contains this version, which is code-wise identical except for the qualifier change in the manifest. org.jboss.tools.runtime.core 2.1.0.Final-v20130717-0450-B102 Where it becomes an issue is that when we start building 4.1.1.Final in November, we will be including both 2.1.0.Final-v20130717-0450-B102 (from JBT 4.1.0.Final) and 2.1.0.Final-v20131200-0000-Bxxx (from JBT 4.1.1.Final). Since these are bitwise identical, there's no point making users update from an identical feature or plugin to a newer identical version. So how do we prevent the build from including a newer-versioned but identical feature or plugin? The only thing I can currently think of is to *exclude* these features from the base_41 update site [3] so that only the older version from [2] will be included. This is basically the process I used to build JBT 4.1.0.1 - I excluded everything from jbosstools-central's site/category.xml except the feature which had actually changed [4]. [4] https://github.com/jbosstools/jbosstools-central/commit/16279020cbf872f4e... Looking for feedback here -- is this a good idea?

I would prefer to have one place where we can configure what is included into release. Aggregate build seems to be a good candidate for this place. Why not just point to exact version 2.1.0.Final-v20130717-0450-B102 of runtime feature in aggregation update site? Denis On 09/27/2013 09:10 AM, Nick Boldt wrote: > In looking at the JBT 4.1.1.Alpha2 build [1], I see that some features > have not yet changed since JBT 4.1.0.Final [2]. This is fine for a > project like GWT or Freemarker, where the entire project is versioned > the same way and we can simply include the previous .Final release, but > for a subcomponent of a project, like the runtime component in > jbosstools-base [3], this presents a question. > > [1] http://download.jboss.org/jbosstools/updates/nightly/core/4.1.kepler/ > > [2] http://download.jboss.org/jbosstools/updates/stable/kepler/ > > [3] > http://download.jboss.org/jbosstools/builds/staging/jbosstools-base_41/al... > > Currently, the base build [3] is producing this plugin: > > org.jboss.tools.runtime.core 2.1.0.Alpha2-v20130927-0137-B114 > > But the JBT 4.1.1.Alpha2 build [1] contains this version, which is > code-wise identical except for the qualifier change in the manifest. > > org.jboss.tools.runtime.core 2.1.0.Final-v20130717-0450-B102 > > Where it becomes an issue is that when we start building 4.1.1.Final in > November, we will be including both 2.1.0.Final-v20130717-0450-B102 > (from JBT 4.1.0.Final) and 2.1.0.Final-v20131200-0000-Bxxx (from JBT > 4.1.1.Final). Since these are bitwise identical, there's no point making > users update from an identical feature or plugin to a newer identical > version. > > So how do we prevent the build from including a newer-versioned but > identical feature or plugin? > > The only thing I can currently think of is to *exclude* these features > from the base_41 update site [3] so that only the older version from [2] > will be included. > > This is basically the process I used to build JBT 4.1.0.1 - I excluded > everything from jbosstools-central's site/category.xml except the > feature which had actually changed [4]. > > [4] > https://github.com/jbosstools/jbosstools-central/commit/16279020cbf872f4e... > > Looking for feedback here -- is this a good idea? > > _______________________________________________ jbosstools-dev mailing list jbosstools-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jbosstools-dev

On 09/30/2013 11:17 AM, Max Rydahl Andersen wrote: >+1 >This is similar to what JBTIS does. This also requires to have components pushing a version they want to get aggregated. This version should be moved to a stable URL, and components owner should provide PR on the aggregator category.xml to use the right URL and the right version.

Aggregator shouldn't consume the "continuous" composite site.

On 09/30/2013 11:50 AM, Max Rydahl Andersen wrote: >i'm leaning more and more towards the continous composite site is >causing more damage and hiding more issues >than it is worth having. The big benfit of this continuous composite site is what you describe here: >[...]some components (such as JST) probably have too high a >dependency on Common in base to make it feasible...but if we could >have these components explicitly point to a "snapshot" of the site >(not the composited site) until release time this might be doable.

We'll need standardized URLs, and we'll need each component to define its dependencies in a pom. Then we can go for configuation such as <properties> <buildType>snapshot</buildType> <!-- Can be snapshot, integration, release --> </properties> <repository> <layout>p2</layout> <url>http://download.jboss.org/jbosstools/updates/jbosstools-base/1.4.0... </repository> The URL pattern needs to control required version (set in pom), and buildType (default value in pom, overridable via command-line).

We can't easily use a target definition for that as they don't allow properties.

-- Mickael Istria Eclipse developer at JBoss, by Red Hat <http://www.jboss.org/tools> My blog <http://mickaelistria.wordpress.com> - My Tweets <http://twitter.com/mickaelistria>

HOWEVER... That doesn't solve the issue of having a subcomponent *unchanged* when the rest of the component *HAS changed*.

In that case, we would need to release the unchanged subcomponents separate from the changed ones, so that we can composite/aggregate the correct versions and avoid the impending situation described below, where an end-user updating from JBT 4.1.0 to 4.1.1 would get two copies of the same plugin: org.jboss.tools.runtime.core 2.1.0.Final-v20130717-0450-B102 (in 4.1.0) org.jboss.tools.runtime.core 2.1.0.Final-v20131200-0000-Bxxx (in 4.1.1)

I would prefer to not start adding more github repos and jobs

, but if these subcomponents are going to evolve more slowly than their containing projects, then maybe that's a necessary evil? Otherwise, we could simply have more than one update site produced by the project,

and selectively publish them to different locations so the unchanged can be released to /updates/stable/kepler/<project>/<subcomponent>/<version> and the changing can remain in CI / staging, to be aggregated along with the rest of the stack.

On 09/30/2013 05:17 AM, Max Rydahl Andersen wrote: >+1 > >This is similar to what JBTIS does. > >/max > >On Fri, Sep 27, 2013 at 10:28:36AM -0700, Denis Golovin wrote: >>I would prefer to have one place where we can configure what is included >>into release. >>Aggregate build seems to be a good candidate for this place. Why not >>just point to exact version >>2.1.0.Final-v20130717-0450-B102 of runtime feature in aggregation update >>site? >> >>Denis >> >>On 09/27/2013 09:10 AM, Nick Boldt wrote: >>>In looking at the JBT 4.1.1.Alpha2 build [1], I see that some features >>>have not yet changed since JBT 4.1.0.Final [2]. This is fine for a >>>project like GWT or Freemarker, where the entire project is versioned >>>the same way and we can simply include the previous .Final release, but >>>for a subcomponent of a project, like the runtime component in >>>jbosstools-base [3], this presents a question. >>> >>>[1] >>>http://download.jboss.org/jbosstools/updates/nightly/core/4.1.kepler/ >>> >>>[2] http://download.jboss.org/jbosstools/updates/stable/kepler/ >>> >>>[3] >>>http://download.jboss.org/jbosstools/builds/staging/jbosstools-base_41/all/repo/ >>> >>> >>>Currently, the base build [3] is producing this plugin: >>> >>>org.jboss.tools.runtime.core 2.1.0.Alpha2-v20130927-0137-B114 >>> >>>But the JBT 4.1.1.Alpha2 build [1] contains this version, which is >>>code-wise identical except for the qualifier change in the manifest. >>> >>>org.jboss.tools.runtime.core 2.1.0.Final-v20130717-0450-B102 >>> >>>Where it becomes an issue is that when we start building 4.1.1.Final in >>>November, we will be including both 2.1.0.Final-v20130717-0450-B102 >>>(from JBT 4.1.0.Final) and 2.1.0.Final-v20131200-0000-Bxxx (from JBT >>>4.1.1.Final). Since these are bitwise identical, there's no point making >>>users update from an identical feature or plugin to a newer identical >>>version. >>> >>>So how do we prevent the build from including a newer-versioned but >>>identical feature or plugin? >>> >>>The only thing I can currently think of is to *exclude* these features >>>from the base_41 update site [3] so that only the older version from [2] >>>will be included. >>> >>>This is basically the process I used to build JBT 4.1.0.1 - I excluded >>>everything from jbosstools-central's site/category.xml except the >>>feature which had actually changed [4]. >>> >>>[4] >>>https://github.com/jbosstools/jbosstools-central/commit/16279020cbf872f4eac59a36a77df01edb2ed832 >>> >>> >>>Looking for feedback here -- is this a good idea? >>> >>> >> >>_______________________________________________ >>jbosstools-dev mailing list &gt;&gt;jbosstools-dev(a)lists.jboss.org >>https://lists.jboss.org/mailman/listinfo/jbosstools-dev -- Nick Boldt :: JBoss by Red Hat Productization Lead :: JBoss Tools & Dev Studio http://nick.divbyzero.com

On 09/27/2013 06:10 PM, Nick Boldt wrote: >The only thing I can currently think of is to *exclude* these features >from the base_41 update site [3] so that only the older version from [2] >will be included. > >This is basically the process I used to build JBT 4.1.0.1 - I excluded >everything from jbosstools-central's site/category.xml except the >feature which had actually changed [4]. I don't find it very easy to deal with picked dependencies on the producer's side. Choosing dependencies is IMO a responsibility of the consumer (JBT aggregator). I tend to agree with Denis when he says that this should be something set in aggregator. We should ask component leads which version of the feature has to be aggregated (last build from branch, or last release?). However, that also shows that all versions should be bumped after a release, and this should be component's developer responsibility: when you have x.y.z released, then immediate next commit is to move to x.y.z+1 or x.y+1.0. Preparing versions for next stream is the last step of a release, not the first of the next release, so it should be done immediatly to prevent such issues. If we do that, then we can filter using version ranges in our aggregator category.xml, which seems to be a very good trade-off between control and flexibility.

It appears that this use-case is covered by Tycho baseline replacement mechanism. This mechanism currently can't work for us until we have reproducible qualifiers (which I don't like).

On 09/30/2013 11:20 AM, Max Rydahl Andersen wrote: >not sure I follow you fully here - which category.xml and what kind >of version ranges would you set ? I'm saying that bundles and features version should be bumped just after a release, and that aggregator category.xml should use version range that match the stream [x.y.z,x.y.z+1) . So we control what stream gets in.

>>It appears that this use-case is covered by Tycho baseline >>replacement mechanism. This mechanism currently can't work for us >>until we have reproducible qualifiers (which I don't like). >Because you want CI == Release builds, right ? Thats just not very >feasible wit p2/tycho :/ No, it's just that it would be better if we would avoid creating new bundles just because build date is different. That's the purpose of whole baseline mechanism, re-use existing compatible builds from release when it makes sense instead of creating the same bundle with another timestamp.

But I think it is indeed not necessary if components have a string versioning and bump their versions after each release. In such case, having a baseline comparator such a the version watch tool would be enough. Rule would be: if same x.y.z than an existing release, but with different qualifier, fail build telling developer to bump the version.

If the developer does not commit anything to that repo, then it does not need a version bump.

The version bump should be done in parallel with the very first change to the repo. If the user fixes a typo, and the component has not yet been version-bumped, then it must now be version-bumped, during that very first commit. But of course repo-owners have the right to simply bump everything immediately after the branch if they feel safer doing it that way... if they feel like they need to so they don't forget to do it later.

> One day, developers will have to deal with something like this: > http://blog.osgi.org/2013/09/baselining-semantic-versioning-made-easy.html > , which is quite strict. This is what we in theory already should be doing - but yes, because we tend to release all together we haven't seen the impact of when it was forgotten.

In looking at the JBT 4.1.1.Alpha2 build [1], I see that some features have not yet changed since JBT 4.1.0.Final [2]. This is fine for a project like GWT or Freemarker, where the entire project is versioned the same way and we can simply include the previous .Final release, but for a subcomponent of a project, like the runtime component in jbosstools-base [3], this presents a question. [1] http://download.jboss.org/jbosstools/updates/nightly/core/4.1.kepler/ [2] http://download.jboss.org/jbosstools/updates/stable/kepler/ [3] http://download.jboss.org/jbosstools/builds/staging/jbosstools-base_41/al... Currently, the base build [3] is producing this plugin: org.jboss.tools.runtime.core 2.1.0.Alpha2-v20130927-0137-B114 But the JBT 4.1.1.Alpha2 build [1] contains this version, which is code-wise identical except for the qualifier change in the manifest. org.jboss.tools.runtime.core 2.1.0.Final-v20130717-0450-B102 Where it becomes an issue is that when we start building 4.1.1.Final in November, we will be including both 2.1.0.Final-v20130717-0450-B102 (from JBT 4.1.0.Final) and 2.1.0.Final-v20131200-0000-Bxxx (from JBT 4.1.1.Final). Since these are bitwise identical, there's no point making users update from an identical feature or plugin to a newer identical version. So how do we prevent the build from including a newer-versioned but identical feature or plugin? The only thing I can currently think of is to *exclude* these features from the base_41 update site [3] so that only the older version from [2] will be included. This is basically the process I used to build JBT 4.1.0.1 - I excluded everything from jbosstools-central's site/category.xml except the feature which had actually changed [4]. [4] https://github.com/jbosstools/jbosstools-central/commit/16279020cbf872f4e... Looking for feedback here -- is this a good idea? -- Nick Boldt :: JBoss by Red Hat Productization Lead :: JBoss Tools & Dev Studio http://nick.divbyzero.com _______________________________________________ jbosstools-dev mailing list jbosstools-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jbosstools-dev