[wildfly-dev] Srcdeps in WildFly and WildFly Core

[Petr Sakar]

On 01/26/2017 09:54 AM, Peter Palaga wrote:
> Hi Brian, thanks for your comments, more inline...
>
> On 2017-01-26 02:02, Brian Stansberry wrote:
>> My only concerns with this would relate to comitting this kind of src
>> dependency to the poms in the main branches in the widlfly/wildfly
>> and wildfly/wildfly-core repos. We’ve managed to survive up to now
>> with little or no need for that kind of thing, so until we get used
>> to using this in other ways IMHO we should follow the KISS principle
>> and forbid that.
> Maybe I overestimate the amount of changes that span over multiple git
> repos. Maybe you in the Core team do not do this often. But for us in
> the Sustaining Engineering Team, this is quite a typical situation. A
> substantial part of the reports from customers come with a description
> how to reproduce on the whole server, but they need to be fixed in a
> component. Having srcdeps would make the CP process simpler and faster,
> allowing us to uncover the conflicts and regressions earlier.
>
>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>> a SRC dependency to get CI or review and accidentally gets merged.
> Oh, I am just realizing I have not said anything about merging. I
> actually do want to propose that commits with source dependencies get
> merged to e.g. wildfly-core master as early as possible. Those are the
> key points of Continuous Integration: get feedback quickly, and merge as
> soon as possible. This is exactly what Hawkular is doing since more than
> a year.

I agree with Brian. PR with SRC dependency *should not* be merged to master branch, that would create problems down the road.
You can use other branch as base for such merges and related CI
Petr

>> But I suppose that’s not the end of the world, so long as the release
>> process will eventually detect it and fail.
> Yes, source dependencies on a stable branch do not harm. They just need
> to be avoided in releases (for which srcdeps offers technical means).
>
>> Can making srcdeps fail (or just disabling it) be turned on via a
>> maven profile? With that we could set up such a profile and turn it
>> on in CI jobs that are testing branches where it’s forbidden (e.g.
>> the nightly builds of master.)
> Yes, the feature is called "failWith profiles" and can be configured in
> .mvn/srcdeps.yaml, like here in this srcdeps quickstart:
> https://github.com/srcdeps/srcdeps-maven/blob/master/srcdeps-maven-quickstarts/srcdeps-mvn-git-profile-quickstart/.mvn/srcdeps.yaml#L33
>
> There is also "failWith properties" and "failWith goals". It is
> documented here:
> https://github.com/srcdeps/srcdeps-core/blob/master/doc/srcdeps.yaml#L130
> By default there is failWith: {goals: release:prepare, release:perform}.
> Projects that do not use the release plugin can set e.g. failWith:
> {goals: deploy:deploy} or whatever else distinguishes their releases.
>
>> Oh, one other concern — how robust is this in the face of poor
>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml.
> Which parts are boilerpate?
>
>> If
>> that gets out of date or something is the only effect that using a
>> src dependency for the affected item doesn't work?
> Yes, I think so. As long as the .mvn/srcdeps.yaml file is syntactically
> correct, any misconfiguration there should not have any other effect
> than eventually breaking an embedded build.
>
> Generally, the things configured in .mvn/srcdeps.yaml tend to be quite
> stable - it is basically just mapping from GAVs to their respective git
> URLs. Git URLs do not change often. It is true that dependency artifacts
> come and go, but as long as their groupIds are selected reasonably (one
> groupId occurs in not more than one git repo) the mapping itself can be
> quite stable over time too.
>
> Thanks,
>
> Peter
>
>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com>
>>> wrote:
>>>
>>> Hi *,
>>>
>>> this is not new to those of you who attended my talk on the F2F
>>> 2016 in Brno. Let me explain the idea here again for all others who
>>> did not have a chance to be there.
>>>
>>> Srcdeps [1] is a tool to build Maven dependencies from their
>>> sources. With srcdeps, wildfly-core can depend on a specific commit
>>> of, e.g., undertow:
>>>
>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>
>>>
>>>
> where aabbccd is the git commit id to build when any undertow artifact
>>> is requested during the build of wildfly-core.
>>>
>>> [1] describes in detail, how it works.
>>>
>>> The main advantage of srcdeps is that changes in components can be
>>>   integrated and tested in wildfly-core immediately after they are
>>> committed to a public component branch. There is no need to wait
>>> for the component release.
>>>
>>> Here in the WildFly family of projects, it is often the case that
>>> something needs to be fixed in a component, but the verification
>>> (using bug reproducer, or integration test) is possible only at the
>>> level of wildfly or wildfly-core. Engineers typically work with
>>> snapshots locally, but when their changes need to get shared (CI,
>>> reviews) in a reproducible manner, snapshots cannot be used
>>> anymore. In such situations a source dependency come in handy: it
>>> is very easy to share and it is as reproducible as a Maven build
>>> from a specific commit can be. All CIs and reviewers can work with
>>> it, because all source dependency compilation is done under the
>>> hood by Maven.
>>>
>>> Developers working on changes that span over multiple
>>> interdependent git repos can thus get feedback (i-tests, reviews)
>>> quickly without waiting for releases of components.
>>>
>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>> this kind of situation and is in use there since around October
>>> 2015.
>>>
>>> When I said there is no need to wait for releases of components, I
>>> did not mean that we can get rid of component releases altogether.
>>> Clearly, we cannot, because i.a. for any tooling uninformed about
>>> how srcdeps work, those source dependencies would simply be
>>> non-resolvable from public Maven repositories. So, before releasing
>>> the dependent component (such as wildfly-core) all its dependencies
>>> need to be released. To enforce this, srcdeps is by default
>>> configured to make the release fail, as long as there are source
>>> dependencies.
>>>
>>> I have sent a PR introducing srcdeps to wildfly-core:
>>> https://github.com/wildfly/wildfly-core/pull/2122 To get a feeling
>>> how it works, checkout the branch, switch to e.g.
>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>>   (that happens to be the commit id of the 1.4.8.Final tag) and
>>> build wildfly-core as usual with "mvn clean install". You'll see in
>>>   the build log that undertow is being cloned to
>>> ~/.m2/srcdeps/io/undertow and that it is built there. After the
>>> build, check that the
>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>> version of Undertow got installed to your local Maven repo (usually
>>>   ~/m2/repository/io/undertow/undertow-core )
>>>
>>> Are there any questions or comments?
>>>
>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>
>>> Thanks,
>>>
>>> Peter
>>>
>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>> 14:30 at DevConf Brno.
>>> _______________________________________________ wildfly-dev mailing
>>> list wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev