[jbosstools-issues] [JBoss JIRA] (JBIDE-21145) Composite install job does not reliably install all JBT IUs and so does not see changes between CI builds

[Nick Boldt (JIRA)]

    [ https://issues.jboss.org/browse/JBIDE-21145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138538#comment-13138538 ] 

Nick Boldt commented on JBIDE-21145:
------------------------------------

Max, you're giving me whiplash here. First you agree that since composite install test is a DIFFERENT installation test than the aggregate one, it makes sense to keep it:

"It still makes sense to have composite install tests running since not everyting is in sync for aggregateand the composite"

Then you state it's not necessary based on an incorrect assumption: that they're doing the same thing. Let me repeat. Again. The composite install test is NOT the same as the aggregate install tests.

"I see both me and mistria repeating that the composite install test is unnecessary assuming it does the same as building the aggregate"

As to what all this has to do w/ JBDS, please re-read what I said. We do MANY install tests while building JBT and JBDS. Most are done once everything is built and published to CI, since the install tests require that all the moving pieces stop moving for a little bit. The composite install test is the only one that tests composition of the aggregate BEFORE it's built. So, it's a valuable test. 

If you disagree, you can simply ignore it and let the aggregate jobs run on a schedule, checking for github changes (which they do), or on a schedule rebuilding and then checking buildinfo.json or p2diff to see if a publish is worthwhile (which they do).

If you want more frequent github (Jenkins built-in) or sha/p2diff checks (JBIDE-16970), we can certainly make that change. 

"We could directly run the aggregator every 3 hours."

I'm good with that idea, if there's a need for it. Do we need to rebuild the aggregate every 3 hrs just to see if there's new bits to publish?

As to the value of the composite install test... well, I just proved another case where it's useful. Yesterday I added the Central/EA URLs into the mix of sites from which the install is being done, in order to resolve JBIDE-21189. In 4.3.mars branch, the composite install is now blue/yellow again, instead of failing with:

{code}
   [p2.dir]   From: JBoss Tools Java Standard Tools Tests Developer Resources 3.7.1.Beta1-v201   [p2.dir]  Missing requirement: Tests Plug-in 3.7.1.Beta1-v20151204-1915-B61 (org.jboss.tools.jst.angularjs.test 3.7.1.Beta1-v20151204-1915-B61) requires 'bundle org.eclipse.angularjs.core 1.1.0' but it could not be found
51204-1915-B61 (org.jboss.tools.jst.test.feature.source.feature.group 3.7.1.Beta1-v20151204-1915-B61)
{code}

But in master branch, I'm getting a new problem -- there are two different versions of org.eclipse.mylyn.context_feature - 3.17.0.v20150625-2044 and 3.18.0.v20151014-2324 - in our target platforms. So, once again, the composite install job finds another problem! [^jbide21145-conflict-between-tp-and-central.png]

So, *please* can we stop bitching about its value? We do this every year, but haven't managed to to find (or build) a BETTER way to find build/install problems like this, or to orchestrate parallel build jobs (core aggregate, coretests aggregate). 



> Composite install job does not reliably install all JBT IUs and so does not see changes between CI builds
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: JBIDE-21145
>                 URL: https://issues.jboss.org/browse/JBIDE-21145
>             Project: Tools (JBoss Tools)
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 4.3.1.Beta1, 4.4.0.Alpha1
>            Reporter: Pavol Srna
>            Assignee: Nick Boldt
>            Priority: Blocker
>             Fix For: 4.3.1.Beta1, 4.4.0.Alpha1
>
>         Attachments: jbide21145-conflict-between-tp-and-central.png
>
>
> We have often seen old artifacts on nightly sites (mars and neon too). 
> It seems that the composite-install job [0], [1] is not reliable. So, the downstream JBT aggregate builds [2], [3] are not triggered automatically to pick up all new changes in the upstream JBT component site builds.
> [0] http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jbosstools-composite-install_master/lastBuild/console
> [1] http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jbosstools-composite-install_4.3.mars/lastBuild/console
> [2] http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jbosstools-build-sites.aggregate.site_master/ (latest build shows: "Nov 27, 2015 6:15 AM NOT ​PUBLISHED: ​​UNCHANGED")
> [3] http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jbosstools-build-sites.aggregate.site_4.3.mars/ (latest build shows: "Nov 27, 2015 3:43 AM NOT ​PUBLISHED: ​​UNCHANGED")
> Please investigate.
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)