[wildfly-dev] Policies for merging PRs on master

Thu Dec 7 02:29:13 EST 2017

Excellent suggestions Rostislav!  There are some trade offs due to the
amount of time needed for each test run.  Whatever the implementation, it
would be good to have a prioritized way of running the tests that
accomplish a-c.

Scott

On Wed, Dec 6, 2017 at 7:31 AM, Rostislav Svoboda <rsvoboda at redhat.com>
wrote:

> I see 3 main use-cases for TCK runs outside WFLY master
>
>  a) pre-checks before final merging to WFLY master - master-ignore way
> discussed here
>  b) checks on big feature(s) development branches
>       something like ladybird or RFE development across multiple components
>       selected TCK modules can be executed, depends on scope of changes
>  c) regular component checks on component master
>       early / regression checks on component level that they do not regress
>       TCK modules related to the component and layered components should
> be executed
>       I know only few components which run related TCK modules with their
> master
>
> With proposed changes for quick WF delivery I believe use-cases b) and c)
> will need to be addressed.
>
> Use-cases b) and c) could be done on component level or via central
> pipeline.
>   component level - prepare automated way to run TCK with custom build -
> e.g. bash script, docker image
>   central pipeline - set of jobs can be prepared and linked via Jenkins
> pipeline so the end user (or curl request) provides just URL of custom WFLY
> build zip + list of modules to be executed
>
> Rostislav
>
> ----- Original Message -----
> > On Tue, Dec 5, 2017 at 4:37 AM, Scott Marlow < smarlow at redhat.com >
> wrote:
> >
> >
> > It would be great if we could have a branch that includes all of the
> commits
> > that we are considering to merge at a particular time of day, such that
> we
> > would run the TCK against that branch, only once a day.
> >
> > Can this be done that often? I had in my mind that if we did one of
> these it
> > would amount to stealing one of the regular runs, but perhaps that's not
> the
> > case.
> >
> > Now, I don't think we'd want to do these anywhere near that often, but
> it's
> > good to know what the limits would be. For example, I could imagine
> doing 10
> > of these over the course of a WF release, but by luck or whatever 3 of
> them
> > come in the same week.
> >
> > +1 to using a branch. We have a branch like that, master-ignore, that we
> use
> > for batching up PRs to test as a group before merging. I wouldn't want to
> > use master-ignore for this, but a differently named branch run the same
> way
> > sounds good.
> >
> >
> >
> > If one of the changes cause a TCK failure, none of them get merged
> > (investigation follows that to determine which change caused the
> > failure(s)), if the test succeeds, we can then merge that batch of
> changes
> > into WildFly master.
> >
> > We likely would want to avoid running the testing, on days when we
> haven't
> > merged any changes to the WF testing branching.
> >
> >
> > Can the TCK be set up to run based on a check for a change in the sha of
> the
> > head of a branch? So every day at a fixed time it checks the branch, and
> > does nothing if there is no change. If we want a run, we force push the
> > branch before that time. We have CI jobs that check master-ignore that
> way,
> > except they poll regularly, not just once a day. That works for those as
> > they aren't so resource intensive that we worry a lot about limiting how
> > often they run.
> >
> >
> > Would that approach help how we merge PRs on master?
> >
> >
> > I think it could be helpful earlier in the release cycle before merging
> big
> > changes, and then perhaps late in the release cycle if we're worried
> about
> > possible regressions.
> >
> >
> >
> > Scott
> >
> > On 12/04/2017 09:33 PM, Stuart Douglas wrote:
> >
> >
> >
> >
> > On Tue, Dec 5, 2017 at 3:40 AM, Brian Stansberry <
> > brian.stansberry at redhat.com <mailto: brian.stansberry at redhat.com >>
> wrote:
> >
> > Great. :)
> >
> > One thing I think we need to do is figure out how to get custom TCK
> > runs for PR branches. The TCK is a big part of our test coverage,
> > and one way to not "use master as a test bed" is to get a check of a
> > branch on the TCK before we merge it.
> >
> > I know we've gotten TCK runs of ad-hoc branches before, so by
> > "figure out" I mean work out how to make that not overly painful,
> > come to some sort of consensus on when it's worthwhile, etc.
> >
> >
> > I think if we were going to do this it should probably be something
> reviewers
> > can ask for on specific PR. The TCK uses a *lot* more resources than a
> > standard CI run, so we need to make sure we limit it to cases where it is
> > required.
> >
> > Stuart
> >
> >
> > On Fri, Dec 1, 2017 at 10:04 AM, Alessio Soldano
> > < asoldano at redhat.com <mailto: asoldano at redhat.com >> wrote:
> >
> > There you go... PR updated to consume the same api jar now
> > released as final.
> >
> > Cheers
> > Alessio
> >
> > On Fri, Dec 1, 2017 at 3:30 PM, David Lloyd
> > < david.lloyd at redhat.com <mailto: david.lloyd at redhat.com >> wrote:
> >
> > On Thu, Nov 30, 2017 at 5:50 PM, Alessio Soldano
> > < asoldano at redhat.com <mailto: asoldano at redhat.com >> wrote:
> > > As suggested by Brian, I'd like to draw attention to the discussion on
> > > https://github.com/wildfly/wildfly/pull/10604
> > < https://github.com/wildfly/wildfly/pull/10604 > .
> > > The PR is an upgrade of the webservices stack, including JBossWS,
> Apache
> > > CXF, JAXB-RI and JAXB API. In particular, the JAXB upgrade is for EE8
> and
> > > better JDK 9 compatibility.
> > > Now, due to the upgrade of the JAXB API spec jar, the PR is essentially
> > > stalled since 20 days; the new spec is released as an alpha (as it's
> been
> > > tested within JBossWS only) and that does not satisfy a rule that
> requires
> > > any artifact being pulled to be Final.
> > > We're talking about a spec jar, we could simply re-tag that as Final,
> > > chances are we won't need changes any time soon there anyway, but as
> Tomaz
> > > pointed out, in principle that would be dishonest.
> >
> > My opinion is that you should go ahead and make a .Final
> > tag. In the
> > (unlikely?) event that the spec has to be modified for some
> > reason, I
> > think you could make a 1.0.1.Final tag and call it a "bug fix".
> >
> > The alternative is to simply wait. I don't think there is
> > any middle position.
> >
> > > While I see the point in requiring that only sufficiently stable
> upgrades
> > > are applied to the codebase, I'm wondering whether, maybe, we're going
> a
> > > bit
> > > too far with the rules. Brian wrote on this topic: "how to determine
> that
> > > something is good enough to go in without using master as a test bed" ?
> >
> > I don't think we are; I agree with the policy as it stands. If you
> > look at it in terms of being able to release at any time,
> > then it
> > follows that everything _must_ be stable.
> >
> > --
> > - DML
> >
> >
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org <mailto: wildfly-dev at lists.jboss.org >
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> > < https://lists.jboss.org/mailman/listinfo/wildfly-dev >
> >
> >
> >
> >
> > -- Brian Stansberry
> > Manager, Senior Principal Software Engineer
> > Red Hat
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org <mailto: wildfly-dev at lists.jboss.org >
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> > < https://lists.jboss.org/mailman/listinfo/wildfly-dev >
> >
> >
> >
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> >
> >
> >
> >
> > --
> > Brian Stansberry
> > Manager, Senior Principal Software Engineer
> > Red Hat
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20171207/734174ce/attachment.html 