[jbosstools-dev] Tests error during local build

[Nick Boldt]

The stated workflow is already being done [0], except that today:

* the current (N) and previous (N-1) are composited together into a 
single site [1]

* the project's previous build is NOT then removed from the composite & 
deleted once the new build is done publishing

* the composite*.xml metadata files are not regenerated when publishing, 
but are *manually* updated if/when a new component is added or job is 
renamed

[0] 
https://github.com/jbosstools/jbosstools-build-ci/blob/master/publish/publish.sh#L541

[1] 
http://download.jboss.org/jbosstools/builds/staging/_composite_/core/trunk/compositeArtifacts.xml

This is done because:

* after the publish of the N build is done, before the bits are visible 
on download.jboss.org, the lag seems to vary from seconds to minutes.

* a build-in-progress which resolved against the N-1 bits will fail if 
those bits suddenly vanish.

---

For the stable branch, the same push-then-rename approach [0] is used 
(rather than destructively pushing on top of an existing build, as we 
did a few years ago), but we only composite [2] the latest (N) because:

* stable branch builds change less often and
* we want to guarantee that we're using the latest.

[2] 
http://download.jboss.org/jbosstools/builds/staging/_composite_/core/4.1.kepler/compositeArtifacts.xml

Hope that makes sense. If you think something can be improved w/o 
causing new problems (such as updating the timestamp in the 
composite*.xml files dynamically w/ every published build, so p2 sees it 
as "new" more often), please don't hesitate to open a JIRA w/ details on 
what you'd change, and how.

Cheers,

Nick

On 05/21/2013 02:12 PM, Denis Golovin wrote:
> On 05/21/2013 10:32 AM, Mickael Istria wrote:
>> On 05/21/2013 06:56 PM, Denis Golovin wrote:
>>> The problem is org.jboss.tools.tests is not part of
>>>
>>> http://download.jboss.org/jbosstools/updates/nightly/core/trunk/plugins/
>> That's right. However it's part of
>> http://download.jboss.org/jbosstools/updates/nightly/integrationtests/trunk
>> site, so when enabling no profile, the default nightly sites (core and
>> integration tests) are used, so it should resolve this bundle.
>> Is this something you can reproduce at every build? It could happen
>> when your build tries to get content at the same time aggregation is
>> getting published.
>
> Pushing process can be implemented different way to avoid this issue, I
> saw it many times and it is a bit annoying. I remember we spend some
> time with Nick to tackle it, but problem seems still here.
>
> Idea behind this fix is really simple:
> 1. Leave old published bits as is on the server side and just upload new
> ones
> 2. Update compositeArtifacts.xml first and include uploaded update site
> from step 1
> 3. Update compositeContent.xml: include new uploaded update site from
> step 1 and remove oldest one
> 4. Update compositeArtifacts.xml and remove oldest one
> 5. Remove oldest update site folder
>
> Note there are no operations related to renaming/moving previously
> uploaded update sites and that the key point to have previously uploaded
> sites available while new one is in uploading stage.
>
> It should significantly reduce amount of errors because we keep two
> update sites for each jbosstools-module, so at least one update site is
> always available through composite update site for module. There could
> be still problems for builds with slow connection, but connection should
> be slow enough to live through at least two builds for jbosstools
> module. This could be implemented as maven plug-n and number of builds
> to keep in composite could be a good candidate for configuration parameters.
>
> It is not really critical but nice to have.
>
> Denis
>
>> --
>> Mickael Istria
>> Eclipse developer at JBoss, by Red Hat <http://www.jboss.org/tools>
>> My blog <http://mickaelistria.wordpress.com> - My Tweets
>> <http://twitter.com/mickaelistria>
>>
>>
>> _______________________________________________
>> jbosstools-dev mailing list
>> jbosstools-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>
>
>
> _______________________________________________
> jbosstools-dev mailing list
> jbosstools-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jbosstools-dev
>

-- 
Nick Boldt :: JBoss by Red Hat
Productization Lead :: JBoss Tools & Dev Studio
http://nick.divbyzero.com