From bbaranow at redhat.com  Tue Jan  3 02:06:11 2017
From: bbaranow at redhat.com (Bartosz Baranowski)
Date: Tue, 3 Jan 2017 02:06:11 -0500 (EST)
Subject: [wildfly-dev] Add a warning if some configuration change has wider
 impact and could cause error?
In-Reply-To: <1256672601.7445801.1483353274321.JavaMail.zimbra@redhat.com>
Message-ID: <309297395.7678675.1483427171321.JavaMail.zimbra@redhat.com>

Hey guys.


In some cases there are/will be conf changes that might require user to make additional change, for instance [1]. 
In this case, if user does not edit the name in two places, when remote client tries to reach out, it will fail.

As far as I remember, there as of now, there is no formal constraint feature for config values.
I've talked with Brian about this and it seems and viable option would be to simply add a header to response.
In CLI, it would be clearly visible, in console, it would require a bit  of hacking to present 
information/hint what needs to be done to configuration to make it work properly.


Any thoughts?

[1] https://issues.jboss.org/browse/WFCORE-1987

From mmusgrov at redhat.com  Thu Jan  5 07:04:09 2017
From: mmusgrov at redhat.com (Michael Musgrove)
Date: Thu, 5 Jan 2017 12:04:09 +0000
Subject: [wildfly-dev] jdk 9 reflection problem
Message-ID: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>

We have just upgraded our jdk build to 9-ea+149 which has tightened up the
accessibility rules. The recommendation seems to be to use the --add-opens
flag to override the rules.

I tried export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED"
which made no difference. Our jdk 9 FAQ does not provided any hints either.

Can anyone provide advice on how to resolve the following maven build error:


[WARNING] Error injecting: org.apache.maven.plugins.war.WarMojo
com.google.inject.ProvisionException: Unable to provision, see the
following errors:

1) Error injecting constructor, java.lang.ExceptionInInitializerError
  at org.apache.maven.plugins.war.WarMojo.<init>(Unknown Source)
  while locating org.apache.maven.plugins.war.WarMojo

...

Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
field private final java.util.Comparator java.util.TreeMap.comparator
accessible: module java.base does not "opens java.util" to unnamed m
odule @669f2128
        at
java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:427)
        at
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:201)
        at
java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:171)
        at java.base/java.lang.reflect.Field.setAccessible(Field.java:165)
        at com.thoughtworks.xstream.core.util.Fields.locate(Fields.java:40)
        at
com.thoughtworks.xstream.converters.collections.TreeMapConverter.<clinit>(TreeMapConverter.java:50)


-- 
Michael Musgrove
Transactions Team
e: mmusgrov at redhat.com
t: +44 191 243 0870

Our mission:To be the catalyst in communities of customers, contributors,
and partners creating better technology the open source way.

Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
(US), Charles Peters (US)

Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael
O'Neill(Ireland)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170105/74715fe3/attachment-0001.html 

From tomaz.cerar at gmail.com  Thu Jan  5 08:40:59 2017
From: tomaz.cerar at gmail.com (=?UTF-8?B?VG9tYcW+IENlcmFy?=)
Date: Thu, 5 Jan 2017 14:40:59 +0100
Subject: [wildfly-dev] jdk 9 reflection problem
In-Reply-To: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
References: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
Message-ID: <CAMquZP5O2M0_Gof4BurCQr+RA9ygmWttziwQA4-y4vm3fHZuKw@mail.gmail.com>

Yeah we need to update FAQ with the latest changes around this.

it is not M2_OPTS that you need to set but in most cases surefire args as
you need this as part of test runtime not maven runtime.
but --add-opens is the flag you are properly using for this case.
It in most cases replaces need for --add-exports flags

I would also recommend adding -Dsun.reflect.debugModuleAccessChecks=true
flag that should help you with reflection issues.

What is your use case exactly? building a war?
Maybe link to project / pom.xml that has this issue?

--
tomaz

On Thu, Jan 5, 2017 at 1:04 PM, Michael Musgrove <mmusgrov at redhat.com>
wrote:

> We have just upgraded our jdk build to 9-ea+149 which has tightened up the
> accessibility rules. The recommendation seems to be to use the --add-opens
> flag to override the rules.
>
> I tried export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED"
> which made no difference. Our jdk 9 FAQ does not provided any hints either.
>
> Can anyone provide advice on how to resolve the following maven build
> error:
>
>
> [WARNING] Error injecting: org.apache.maven.plugins.war.WarMojo
> com.google.inject.ProvisionException: Unable to provision, see the
> following errors:
>
> 1) Error injecting constructor, java.lang.ExceptionInInitializerError
>   at org.apache.maven.plugins.war.WarMojo.<init>(Unknown Source)
>   while locating org.apache.maven.plugins.war.WarMojo
>
> ...
>
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> field private final java.util.Comparator java.util.TreeMap.comparator
> accessible: module java.base does not "opens java.util" to unnamed m
> odule @669f2128
>         at java.base/jdk.internal.reflect.Reflection.
> throwInaccessibleObjectException(Reflection.java:427)
>         at java.base/java.lang.reflect.AccessibleObject.
> checkCanSetAccessible(AccessibleObject.java:201)
>         at java.base/java.lang.reflect.Field.checkCanSetAccessible(
> Field.java:171)
>         at java.base/java.lang.reflect.Field.setAccessible(Field.java:165)
>         at com.thoughtworks.xstream.core.util.Fields.locate(Fields.
> java:40)
>         at com.thoughtworks.xstream.converters.collections.
> TreeMapConverter.<clinit>(TreeMapConverter.java:50)
>
>
> --
> Michael Musgrove
> Transactions Team
> e: mmusgrov at redhat.com
> t: +44 191 243 0870 <+44%20191%20243%200870>
>
> Our mission:To be the catalyst in communities of customers, contributors,
> and partners creating better technology the open source way.
>
> Registered in England and Wales under Company Registration No. 03798903
> Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
> (US), Charles Peters (US)
>
> Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael
> O'Neill(Ireland)
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170105/a0b2164a/attachment.html 

From zfeng at redhat.com  Thu Jan  5 09:01:57 2017
From: zfeng at redhat.com (Amos Feng)
Date: Thu, 5 Jan 2017 09:01:57 -0500 (EST)
Subject: [wildfly-dev] jdk 9 reflection problem
In-Reply-To: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
References: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
Message-ID: <1943667689.8511425.1483624917212.JavaMail.zimbra@redhat.com>

Hi Mike, 

I had tried MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.text=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED" and it works.

Amos

----- Original Message -----
From: "Michael Musgrove" <mmusgrov at redhat.com>
To: "WildFly Dev" <wildfly-dev at lists.jboss.org>
Sent: Thursday, January 5, 2017 8:04:09 PM
Subject: [wildfly-dev] jdk 9 reflection problem

We have just upgraded our jdk build to 9-ea+149 which has tightened up the accessibility rules. The recommendation seems to be to use the --add-opens flag to override the rules. 

I tried export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED" which made no difference. Our jdk 9 FAQ does not provided any hints either. 

Can anyone provide advice on how to resolve the following maven build error: 


[WARNING] Error injecting: org.apache.maven.plugins.war.WarMojo 
com.google.inject.ProvisionException: Unable to provision, see the following errors: 

1) Error injecting constructor, java.lang.ExceptionInInitializerError 
at org.apache.maven.plugins.war.WarMojo.<init>(Unknown Source) 
while locating org.apache.maven.plugins.war.WarMojo 

... 

Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed m 
odule @669f2128 
at java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:427) 
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:201) 
at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:171) 
at java.base/java.lang.reflect.Field.setAccessible(Field.java:165) 
at com.thoughtworks.xstream.core.util.Fields.locate(Fields.java:40) 
at com.thoughtworks.xstream.converters.collections.TreeMapConverter.<clinit>(TreeMapConverter.java:50) 


-- 
Michael Musgrove 
Transactions Team 
e: mmusgrov at redhat.com 
t: +44 191 243 0870 

Our mission:To be the catalyst in communities of customers, contributors, and partners creating better technology the open source way. 

Registered in England and Wales under Company Registration No. 03798903 
Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson 
(US), Charles Peters (US) 

Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael O'Neill(Ireland) 

_______________________________________________
wildfly-dev mailing list
wildfly-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/wildfly-dev

From zfeng at redhat.com  Thu Jan  5 09:07:52 2017
From: zfeng at redhat.com (Amos Feng)
Date: Thu, 5 Jan 2017 09:07:52 -0500 (EST)
Subject: [wildfly-dev] jdk 9 reflection problem
In-Reply-To: <1943667689.8511425.1483624917212.JavaMail.zimbra@redhat.com>
References: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
	<1943667689.8511425.1483624917212.JavaMail.zimbra@redhat.com>
Message-ID: <907785819.8515112.1483625272461.JavaMail.zimbra@redhat.com>

Also it looks like a maven-war-plugin issue https://issues.apache.org/jira/browse/MWAR-397

----- Original Message -----
From: "Amos Feng" <zfeng at redhat.com>
To: "Michael Musgrove" <mmusgrov at redhat.com>
Cc: "WildFly Dev" <wildfly-dev at lists.jboss.org>
Sent: Thursday, January 5, 2017 10:01:57 PM
Subject: Re: [wildfly-dev] jdk 9 reflection problem

Hi Mike, 

I had tried MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.text=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED" and it works.

Amos

----- Original Message -----
From: "Michael Musgrove" <mmusgrov at redhat.com>
To: "WildFly Dev" <wildfly-dev at lists.jboss.org>
Sent: Thursday, January 5, 2017 8:04:09 PM
Subject: [wildfly-dev] jdk 9 reflection problem

We have just upgraded our jdk build to 9-ea+149 which has tightened up the accessibility rules. The recommendation seems to be to use the --add-opens flag to override the rules. 

I tried export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED" which made no difference. Our jdk 9 FAQ does not provided any hints either. 

Can anyone provide advice on how to resolve the following maven build error: 


[WARNING] Error injecting: org.apache.maven.plugins.war.WarMojo 
com.google.inject.ProvisionException: Unable to provision, see the following errors: 

1) Error injecting constructor, java.lang.ExceptionInInitializerError 
at org.apache.maven.plugins.war.WarMojo.<init>(Unknown Source) 
while locating org.apache.maven.plugins.war.WarMojo 

... 

Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed m 
odule @669f2128 
at java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:427) 
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:201) 
at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:171) 
at java.base/java.lang.reflect.Field.setAccessible(Field.java:165) 
at com.thoughtworks.xstream.core.util.Fields.locate(Fields.java:40) 
at com.thoughtworks.xstream.converters.collections.TreeMapConverter.<clinit>(TreeMapConverter.java:50) 


-- 
Michael Musgrove 
Transactions Team 
e: mmusgrov at redhat.com 
t: +44 191 243 0870 

Our mission:To be the catalyst in communities of customers, contributors, and partners creating better technology the open source way. 

Registered in England and Wales under Company Registration No. 03798903 
Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson 
(US), Charles Peters (US) 

Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael O'Neill(Ireland) 

_______________________________________________
wildfly-dev mailing list
wildfly-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/wildfly-dev
_______________________________________________
wildfly-dev mailing list
wildfly-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/wildfly-dev

From mmusgrov at redhat.com  Thu Jan  5 09:08:05 2017
From: mmusgrov at redhat.com (Michael Musgrove)
Date: Thu, 5 Jan 2017 14:08:05 +0000
Subject: [wildfly-dev] jdk 9 reflection problem
In-Reply-To: <1943667689.8511425.1483624917212.JavaMail.zimbra@redhat.com>
References: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
	<1943667689.8511425.1483624917212.JavaMail.zimbra@redhat.com>
Message-ID: <CACKZbaviUmwJCJRa9jh+ZDBPGezAaJWcwvRd31ybc4bhSZg3Vw@mail.gmail.com>

Ah yes it works when I include  java.desktop/java.awt.font

Thanks, but how did you figure out it wanted this one as well?

On Thu, Jan 5, 2017 at 2:01 PM, Amos Feng <zfeng at redhat.com> wrote:

> Hi Mike,
>
> I had tried MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED
> --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens
> java.base/java.text=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED"
> and it works.
>
> Amos
>
> ----- Original Message -----
> From: "Michael Musgrove" <mmusgrov at redhat.com>
> To: "WildFly Dev" <wildfly-dev at lists.jboss.org>
> Sent: Thursday, January 5, 2017 8:04:09 PM
> Subject: [wildfly-dev] jdk 9 reflection problem
>
> We have just upgraded our jdk build to 9-ea+149 which has tightened up the
> accessibility rules. The recommendation seems to be to use the --add-opens
> flag to override the rules.
>
> I tried export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED"
> which made no difference. Our jdk 9 FAQ does not provided any hints either.
>
> Can anyone provide advice on how to resolve the following maven build
> error:
>
>
> [WARNING] Error injecting: org.apache.maven.plugins.war.WarMojo
> com.google.inject.ProvisionException: Unable to provision, see the
> following errors:
>
> 1) Error injecting constructor, java.lang.ExceptionInInitializerError
> at org.apache.maven.plugins.war.WarMojo.<init>(Unknown Source)
> while locating org.apache.maven.plugins.war.WarMojo
>
> ...
>
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> field private final java.util.Comparator java.util.TreeMap.comparator
> accessible: module java.base does not "opens java.util" to unnamed m
> odule @669f2128
> at java.base/jdk.internal.reflect.Reflection.
> throwInaccessibleObjectException(Reflection.java:427)
> at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(
> AccessibleObject.java:201)
> at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:171)
> at java.base/java.lang.reflect.Field.setAccessible(Field.java:165)
> at com.thoughtworks.xstream.core.util.Fields.locate(Fields.java:40)
> at com.thoughtworks.xstream.converters.collections.
> TreeMapConverter.<clinit>(TreeMapConverter.java:50)
>
>
> --
> Michael Musgrove
> Transactions Team
> e: mmusgrov at redhat.com
> t: +44 191 243 0870
>
> Our mission:To be the catalyst in communities of customers, contributors,
> and partners creating better technology the open source way.
>
> Registered in England and Wales under Company Registration No. 03798903
> Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
> (US), Charles Peters (US)
>
> Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael
> O'Neill(Ireland)
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>



-- 
Michael Musgrove
Transactions Team
e: mmusgrov at redhat.com
t: +44 191 243 0870

Our mission:To be the catalyst in communities of customers, contributors,
and partners creating better technology the open source way.

Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
(US), Charles Peters (US)

Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael
O'Neill(Ireland)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170105/6b79bb71/attachment-0001.html 

From eduardo.santanadasilva at gmail.com  Thu Jan  5 12:02:18 2017
From: eduardo.santanadasilva at gmail.com (=?UTF-8?Q?Eduardo_Sant=C2=B4Ana_da_Silva?=)
Date: Thu, 5 Jan 2017 15:02:18 -0200
Subject: [wildfly-dev] jdk 9 reflection problem
In-Reply-To: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
References: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
Message-ID: <CAFVp3Z7zaFqMH07XPswJRF2BmVKN4LFHXJYxi1b=zqy5OpEU+A@mail.gmail.com>

I found this, that could help:

https://cwiki.apache.org/confluence/display/MAVEN/Java+9+-+Jigsaw

com.thoughtworks.xstream xstream

Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
field private final java.util.Comparator java.util.TreeMap.comparator
accessible: module java.base does not "opens java.util" to unnamed module
Happens while
initializing org.apache.maven.plugins.war.util.WebappStructureSerializer
maven-war-plugin


http://stackoverflow.com/questions/41265266/how-to-solve-inaccessibleobjectexception-unable-to-make-member-accessible-m/41265267




2017-01-05 10:04 GMT-02:00 Michael Musgrove <mmusgrov at redhat.com>:

> We have just upgraded our jdk build to 9-ea+149 which has tightened up the
> accessibility rules. The recommendation seems to be to use the --add-opens
> flag to override the rules.
>
> I tried export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED"
> which made no difference. Our jdk 9 FAQ does not provided any hints either.
>
> Can anyone provide advice on how to resolve the following maven build
> error:
>
>
> [WARNING] Error injecting: org.apache.maven.plugins.war.WarMojo
> com.google.inject.ProvisionException: Unable to provision, see the
> following errors:
>
> 1) Error injecting constructor, java.lang.ExceptionInInitializerError
>   at org.apache.maven.plugins.war.WarMojo.<init>(Unknown Source)
>   while locating org.apache.maven.plugins.war.WarMojo
>
> ...
>
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> field private final java.util.Comparator java.util.TreeMap.comparator
> accessible: module java.base does not "opens java.util" to unnamed m
> odule @669f2128
>         at java.base/jdk.internal.reflect.Reflection.
> throwInaccessibleObjectException(Reflection.java:427)
>         at java.base/java.lang.reflect.AccessibleObject.
> checkCanSetAccessible(AccessibleObject.java:201)
>         at java.base/java.lang.reflect.Field.checkCanSetAccessible(
> Field.java:171)
>         at java.base/java.lang.reflect.Field.setAccessible(Field.java:165)
>         at com.thoughtworks.xstream.core.util.Fields.locate(Fields.
> java:40)
>         at com.thoughtworks.xstream.converters.collections.
> TreeMapConverter.<clinit>(TreeMapConverter.java:50)
>
>
> --
> Michael Musgrove
> Transactions Team
> e: mmusgrov at redhat.com
> t: +44 191 243 0870 <+44%20191%20243%200870>
>
> Our mission:To be the catalyst in communities of customers, contributors,
> and partners creating better technology the open source way.
>
> Registered in England and Wales under Company Registration No. 03798903
> Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
> (US), Charles Peters (US)
>
> Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael
> O'Neill(Ireland)
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>



-- 
___________________________
Eduardo Sant'Ana da Silva - Ph.D.
Researcher / IT Consultant
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170105/2f24b6e3/attachment.html 

From eduardo.santanadasilva at gmail.com  Thu Jan  5 12:04:04 2017
From: eduardo.santanadasilva at gmail.com (=?UTF-8?Q?Eduardo_Sant=C2=B4Ana_da_Silva?=)
Date: Thu, 5 Jan 2017 15:04:04 -0200
Subject: [wildfly-dev] jdk 9 reflection problem
In-Reply-To: <1943667689.8511425.1483624917212.JavaMail.zimbra@redhat.com>
References: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
	<1943667689.8511425.1483624917212.JavaMail.zimbra@redhat.com>
Message-ID: <CAFVp3Z7=qcsw8qSESNR39wk8xOwJanDxgnNmwwNRwwh4ooNkPA@mail.gmail.com>

The Amos?s solution is the same related here:

http://stackoverflow.com/questions/41265266/how-to-solve-inaccessibleobjectexception-unable-to-make-member-accessible-m/41265267
....

... the correct fix is to launch the JVM as follows:

# --add-opens has the following syntax: {A}/{package}={B}
java --add-opens java.base/java.lang=ALL-UNNAMED  <<<<<<<<<<<<<<<<<<<<<<<<



2017-01-05 12:01 GMT-02:00 Amos Feng <zfeng at redhat.com>:

> Hi Mike,
>
> I had tried MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED
> --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens
> java.base/java.text=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED"
> and it works.
>
> Amos
>
> ----- Original Message -----
> From: "Michael Musgrove" <mmusgrov at redhat.com>
> To: "WildFly Dev" <wildfly-dev at lists.jboss.org>
> Sent: Thursday, January 5, 2017 8:04:09 PM
> Subject: [wildfly-dev] jdk 9 reflection problem
>
> We have just upgraded our jdk build to 9-ea+149 which has tightened up the
> accessibility rules. The recommendation seems to be to use the --add-opens
> flag to override the rules.
>
> I tried export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED"
> which made no difference. Our jdk 9 FAQ does not provided any hints either.
>
> Can anyone provide advice on how to resolve the following maven build
> error:
>
>
> [WARNING] Error injecting: org.apache.maven.plugins.war.WarMojo
> com.google.inject.ProvisionException: Unable to provision, see the
> following errors:
>
> 1) Error injecting constructor, java.lang.ExceptionInInitializerError
> at org.apache.maven.plugins.war.WarMojo.<init>(Unknown Source)
> while locating org.apache.maven.plugins.war.WarMojo
>
> ...
>
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> field private final java.util.Comparator java.util.TreeMap.comparator
> accessible: module java.base does not "opens java.util" to unnamed m
> odule @669f2128
> at java.base/jdk.internal.reflect.Reflection.
> throwInaccessibleObjectException(Reflection.java:427)
> at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(
> AccessibleObject.java:201)
> at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:171)
> at java.base/java.lang.reflect.Field.setAccessible(Field.java:165)
> at com.thoughtworks.xstream.core.util.Fields.locate(Fields.java:40)
> at com.thoughtworks.xstream.converters.collections.
> TreeMapConverter.<clinit>(TreeMapConverter.java:50)
>
>
> --
> Michael Musgrove
> Transactions Team
> e: mmusgrov at redhat.com
> t: +44 191 243 0870
>
> Our mission:To be the catalyst in communities of customers, contributors,
> and partners creating better technology the open source way.
>
> Registered in England and Wales under Company Registration No. 03798903
> Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
> (US), Charles Peters (US)
>
> Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael
> O'Neill(Ireland)
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>



-- 
___________________________
Eduardo Sant'Ana da Silva - Ph.D.
Researcher / IT Consultant
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170105/86f6398e/attachment.html 

From brian.stansberry at redhat.com  Fri Jan  6 11:24:42 2017
From: brian.stansberry at redhat.com (Brian Stansberry)
Date: Fri, 6 Jan 2017 10:24:42 -0600
Subject: [wildfly-dev] Add a warning if some configuration change has
	wider impact and could cause error?
In-Reply-To: <309297395.7678675.1483427171321.JavaMail.zimbra@redhat.com>
References: <309297395.7678675.1483427171321.JavaMail.zimbra@redhat.com>
Message-ID: <DB647D42-81E9-4C68-9B90-74C023C0B160@redhat.com>

Any opinions? This is basically a proposal to provide a formal way of including warning data in operations responses. 

If we did it it would probably be something like:

{
  ?outcome?=>?success?,
  ?result?=>?foo?,
  ?response-headers?=> {
    ?warnings?=[
        ?blah blah blah?,
       { ?some_complex_issue: details?>[?x?,?y?,?z?]}
    ]
  }
}

So a standard response header whose value is a list of ModelNode, different messages using different node structures, but the norm being a simple string.

Probably OperationContext would need to add a method addResponseWarning to take the details of constructing the header out of the hands of the handler authors.

We?d have to make sure these headers propogate properly through the domain, so the effort is more than simply getting standalone.

If the operation is a composite, there are no per-step response headers so the warning should not assume that the context of the warning is obvious. Warning authors forgetting that point is a possible source of UX issues.

Is this worth the effort? This is driven by this one WFCORE-1987 issue, which is a bit unusual and in an ideal world would go away. But here?s the general pattern involved:

2 attributes in separate parts of the model should be consistent or you wil get bad runtime behavior
User can use a CLI batch to set them both atomically, but web console users can?t do that, so even if the user will do the right thing there will be a moment where they are inconsistent
So, failing is not good (forces users to use CLI + batch)
For the same reason a WARN in the log is not great, as unrelated users may be required to react to log warns, and will be annoyed by seeing them just because their colleague didn?t use a batch

Is that last statement enough reason to add this functionality vs just using the server log to warn, or just not warning at all?

In theory we could use the same facility to report management API deprecation warnings back. Doing that might be annoying though. Those are a less serious problem than this WFCORE-1987 thing.


> On Jan 3, 2017, at 1:06 AM, Bartosz Baranowski <bbaranow at redhat.com> wrote:
> 
> Hey guys.
> 
> 
> In some cases there are/will be conf changes that might require user to make additional change, for instance [1]. 
> In this case, if user does not edit the name in two places, when remote client tries to reach out, it will fail.
> 
> As far as I remember, there as of now, there is no formal constraint feature for config values.
> I've talked with Brian about this and it seems and viable option would be to simply add a header to response.
> In CLI, it would be clearly visible, in console, it would require a bit  of hacking to present 
> information/hint what needs to be done to configuration to make it work properly.
> 
> 
> Any thoughts?
> 
> [1] https://issues.jboss.org/browse/WFCORE-1987
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat





From jmesnil at redhat.com  Fri Jan  6 11:48:11 2017
From: jmesnil at redhat.com (Jeff Mesnil)
Date: Fri, 6 Jan 2017 17:48:11 +0100
Subject: [wildfly-dev] Add a warning if some configuration change has
 wider impact and could cause error?
In-Reply-To: <DB647D42-81E9-4C68-9B90-74C023C0B160@redhat.com>
References: <309297395.7678675.1483427171321.JavaMail.zimbra@redhat.com>
	<DB647D42-81E9-4C68-9B90-74C023C0B160@redhat.com>
Message-ID: <E03C1DB7-414F-4D2F-9BB9-F84493D9FC8C@redhat.com>


> On 6 Jan 2017, at 17:24, Brian Stansberry <brian.stansberry at redhat.com> wrote:
> 
> Any opinions? This is basically a proposal to provide a formal way of including warning data in operations responses. 
> 
> If we did it it would probably be something like:
> 
> {
>  ?outcome?=>?success?,
>  ?result?=>?foo?,
>  ?response-headers?=> {
>    ?warnings?=[
>        ?blah blah blah?,
>       { ?some_complex_issue: details?>[?x?,?y?,?z?]}
>    ]
>  }
> }

We had a similar discussion while implementing :migrate operations for legacy subsystems and ended up using a "notification-warnings? parameter in the result node (as opposed to the response-headers node).

> 
> Is that last statement enough reason to add this functionality vs just using the server log to warn, or just not warning at all?

It?s nice to have for sure and would help having a consistent reporting systems in our management clients (we can imagine having flash warnings in the web console and special text in the CLI written to syserr for example).

In an ideal world, I think that any WARN logged in the server console that is caused by a management operation should be reported to the invoker. 
I?d go a step further to constrain the warnings model as a dictionary where each key is a registered logger ID (e.g. XNIO001007) and a value a i18ned string (or a more complex i18ned structure).

Something like:

{
 ?outcome?=>?success?,
 ?result?=>?foo?,
 ?response-headers?=> {
   ?warnings? => {
       ?XNIO00107? => ?some thing happens?,
       ?WFCORE1234? => { ?some_complex_issue? => [?x?,?y?,?z?]}
   }
 }
}

That?d be very helpful to build our knowledge base and report issues.

> In theory we could use the same facility to report management API deprecation warnings back. Doing that might be annoying though. Those are a less serious problem than this WFCORE-1987 thing.

If you go that way, you could somehow configure the level of reported warnings and let the admin configure the level of reported warnings:

addResponseWarning(String id, int level, String)
addResponseWarning(String id, int level, ModelNode data)

We now have a nice new core-management subsystem where the admin could change the reported warning level (defaulting to something corresponding to serious warnings, as opposed to minor warnings such as API deprecation).

jeff

-- 
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/



From david.lloyd at redhat.com  Fri Jan  6 11:58:41 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Fri, 6 Jan 2017 10:58:41 -0600
Subject: [wildfly-dev] jdk 9 reflection problem
In-Reply-To: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
References: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
Message-ID: <3f2a6282-06ed-9bfe-d548-18551bf2000b@redhat.com>

I guess the obvious suggestion is, report a bug to XStream and tell them 
to use the comparator() method on TreeMap instead of poking the field.

Reflecting on JDK classes is going to become pretty hard in Java 9 
without using Unsafe.

On 01/05/2017 06:04 AM, Michael Musgrove wrote:
> We have just upgraded our jdk build to 9-ea+149 which has tightened up
> the accessibility rules. The recommendation seems to be to use the
> --add-opens flag to override the rules.
>
> I tried export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED"
> which made no difference. Our jdk 9 FAQ does not provided any hints either.
>
> Can anyone provide advice on how to resolve the following maven build error:
>
>
> [WARNING] Error injecting: org.apache.maven.plugins.war.WarMojo
> com.google.inject.ProvisionException: Unable to provision, see the
> following errors:
>
> 1) Error injecting constructor, java.lang.ExceptionInInitializerError
>   at org.apache.maven.plugins.war.WarMojo.<init>(Unknown Source)
>   while locating org.apache.maven.plugins.war.WarMojo
>
> ...
>
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> field private final java.util.Comparator java.util.TreeMap.comparator
> accessible: module java.base does not "opens java.util" to unnamed m
> odule @669f2128
>         at
> java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:427)
>         at
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:201)
>         at
> java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:171)
>         at java.base/java.lang.reflect.Field.setAccessible(Field.java:165)
>         at com.thoughtworks.xstream.core.util.Fields.locate(Fields.java:40)
>         at
> com.thoughtworks.xstream.converters.collections.TreeMapConverter.<clinit>(TreeMapConverter.java:50)
>
>
> --
> Michael Musgrove
> Transactions Team
> e: mmusgrov at redhat.com <mailto:mmusgrov at redhat.com>
> t: +44 191 243 0870
>
> Our mission:To be the catalyst in communities of customers,
> contributors, and partners creating better technology the open source way.
>
> Registered in England and Wales under Company Registration No. 03798903
> Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
> (US), Charles Peters (US)
>
> Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael
> O'Neill(Ireland)
>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>

-- 
- DML

From brian.stansberry at redhat.com  Fri Jan  6 12:07:51 2017
From: brian.stansberry at redhat.com (Brian Stansberry)
Date: Fri, 6 Jan 2017 11:07:51 -0600
Subject: [wildfly-dev] Add a warning if some configuration change has
	wider impact and could cause error?
In-Reply-To: <E03C1DB7-414F-4D2F-9BB9-F84493D9FC8C@redhat.com>
References: <309297395.7678675.1483427171321.JavaMail.zimbra@redhat.com>
	<DB647D42-81E9-4C68-9B90-74C023C0B160@redhat.com>
	<E03C1DB7-414F-4D2F-9BB9-F84493D9FC8C@redhat.com>
Message-ID: <95A52809-6D9F-4B09-86DF-9FD134FEE487@redhat.com>


> On Jan 6, 2017, at 10:48 AM, Jeff Mesnil <jmesnil at redhat.com> wrote:
> 
>> 
>> On 6 Jan 2017, at 17:24, Brian Stansberry <brian.stansberry at redhat.com> wrote:
>> 
>> Any opinions? This is basically a proposal to provide a formal way of including warning data in operations responses. 
>> 
>> If we did it it would probably be something like:
>> 
>> {
>> ?outcome?=>?success?,
>> ?result?=>?foo?,
>> ?response-headers?=> {
>>   ?warnings?=[
>>       ?blah blah blah?,
>>      { ?some_complex_issue: details?>[?x?,?y?,?z?]}
>>   ]
>> }
>> }
> 
> We had a similar discussion while implementing :migrate operations for legacy subsystems and ended up using a "notification-warnings? parameter in the result node (as opposed to the response-headers node).
> 

Yes. I think that was the right approach there, as those warnings were a pretty fundamental part of the result of the operation.

>> 
>> Is that last statement enough reason to add this functionality vs just using the server log to warn, or just not warning at all?
> 
> It?s nice to have for sure and would help having a consistent reporting systems in our management clients (we can imagine having flash warnings in the web console and special text in the CLI written to syserr for example).
> 
> In an ideal world, I think that any WARN logged in the server console that is caused by a management operation should be reported to the invoker. 

I was going to say that the private API usage warnings we log couldn?t fit nicely, as they are done via MSC threads that don?t know about any management op. But thinking a bit more I think that?s probably solvable.

> I?d go a step further to constrain the warnings model as a dictionary where each key is a registered logger ID (e.g. XNIO001007) and a value a i18ned string (or a more complex i18ned structure).
> 
> Something like:
> 
> {
> ?outcome?=>?success?,
> ?result?=>?foo?,
> ?response-headers?=> {
>   ?warnings? => {
>       ?XNIO00107? => ?some thing happens?,
>       ?WFCORE1234? => { ?some_complex_issue? => [?x?,?y?,?z?]}
>   }
> }
> }
> 
> That?d be very helpful to build our knowledge base and report issues.
> 

It would be interesting if there were a nice way to integrate such keys with our existing i18n message infrastructure. Really we?d have to do that, or else have the keys unambiguously different from the message ids.

>> In theory we could use the same facility to report management API deprecation warnings back. Doing that might be annoying though. Those are a less serious problem than this WFCORE-1987 thing.
> 
> If you go that way, you could somehow configure the level of reported warnings and let the admin configure the level of reported warnings:
> 
> addResponseWarning(String id, int level, String)
> addResponseWarning(String id, int level, ModelNode data)
> 
> We now have a nice new core-management subsystem where the admin could change the reported warning level (defaulting to something corresponding to serious warnings, as opposed to minor warnings such as API deprecation).
> 

It could be controllable via a request-header as well, allowing clients to declare what they want, overriding any server-side settings..

> jeff
> 
> -- 
> Jeff Mesnil
> JBoss, a division of Red Hat
> http://jmesnil.net/

-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat





From mmusgrov at redhat.com  Fri Jan  6 13:53:05 2017
From: mmusgrov at redhat.com (Michael Musgrove)
Date: Fri, 6 Jan 2017 18:53:05 +0000
Subject: [wildfly-dev] jdk 9 reflection problem
In-Reply-To: <3f2a6282-06ed-9bfe-d548-18551bf2000b@redhat.com>
References: <CACKZbavNv7Zv4bWCiSHh4ky0i4ve387yZyjhaybStEdma6nQHw@mail.gmail.com>
	<3f2a6282-06ed-9bfe-d548-18551bf2000b@redhat.com>
Message-ID: <CACKZbas8Q5CViRYsfY8giaR5au8W9bon79p7ZewaeyAwAeyO+A@mail.gmail.com>

Good suggestion David. I have raised the issue with XStream (
https://groups.google.com/forum/#!topic/xstream-user/sOHazgzxU2o)

On Fri, Jan 6, 2017 at 4:58 PM, David M. Lloyd <david.lloyd at redhat.com>
wrote:

> I guess the obvious suggestion is, report a bug to XStream and tell them
> to use the comparator() method on TreeMap instead of poking the field.
>
> Reflecting on JDK classes is going to become pretty hard in Java 9
> without using Unsafe.
>
> On 01/05/2017 06:04 AM, Michael Musgrove wrote:
> > We have just upgraded our jdk build to 9-ea+149 which has tightened up
> > the accessibility rules. The recommendation seems to be to use the
> > --add-opens flag to override the rules.
> >
> > I tried export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED"
> > which made no difference. Our jdk 9 FAQ does not provided any hints
> either.
> >
> > Can anyone provide advice on how to resolve the following maven build
> error:
> >
> >
> > [WARNING] Error injecting: org.apache.maven.plugins.war.WarMojo
> > com.google.inject.ProvisionException: Unable to provision, see the
> > following errors:
> >
> > 1) Error injecting constructor, java.lang.ExceptionInInitializerError
> >   at org.apache.maven.plugins.war.WarMojo.<init>(Unknown Source)
> >   while locating org.apache.maven.plugins.war.WarMojo
> >
> > ...
> >
> > Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> > field private final java.util.Comparator java.util.TreeMap.comparator
> > accessible: module java.base does not "opens java.util" to unnamed m
> > odule @669f2128
> >         at
> > java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectExcepti
> on(Reflection.java:427)
> >         at
> > java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(
> AccessibleObject.java:201)
> >         at
> > java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:171)
> >         at java.base/java.lang.reflect.Field.setAccessible(Field.
> java:165)
> >         at com.thoughtworks.xstream.core.util.Fields.locate(Fields.
> java:40)
> >         at
> > com.thoughtworks.xstream.converters.collections.
> TreeMapConverter.<clinit>(TreeMapConverter.java:50)
> >
> >
> > --
> > Michael Musgrove
> > Transactions Team
> > e: mmusgrov at redhat.com <mailto:mmusgrov at redhat.com>
> > t: +44 191 243 0870
> >
> > Our mission:To be the catalyst in communities of customers,
> > contributors, and partners creating better technology the open source
> way.
> >
> > Registered in England and Wales under Company Registration No. 03798903
> > Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
> > (US), Charles Peters (US)
> >
> > Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael
> > O'Neill(Ireland)
> >
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> >
>
> --
> - DML
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>



-- 
Michael Musgrove
Transactions Team
e: mmusgrov at redhat.com
t: +44 191 243 0870

Our mission:To be the catalyst in communities of customers, contributors,
and partners creating better technology the open source way.

Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
(US), Charles Peters (US)

Michael Cunningham (US), Charles Peters (US), Matt Parson (US), Michael
O'Neill(Ireland)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170106/1bbd26bc/attachment.html 

From gaoyonglu at 126.com  Sun Jan  8 22:54:29 2017
From: gaoyonglu at 126.com (=?GBK?B?uN/TwMK2?=)
Date: Mon, 9 Jan 2017 11:54:29 +0800 (CST)
Subject: [wildfly-dev] About the function of StuckThreadMaxTime to add
Message-ID: <41eb1380.467a.159815cf1c1.Coremail.gaoyonglu@126.com>

StuckThreadMaxTime: Enter the number of seconds, that a thread must be continually working before this server diagnoses the thread as being stuck.When time is up?the thread will killed?
In weblogic is 600s.I think this function useful for some scenarios?
Wildfly is consider adding this feature?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170109/2666d2bb/attachment-0001.html 

From bbaranow at redhat.com  Mon Jan  9 01:55:52 2017
From: bbaranow at redhat.com (Bartosz Baranowski)
Date: Mon, 9 Jan 2017 01:55:52 -0500 (EST)
Subject: [wildfly-dev] Add a warning if some configuration change has
 wider impact and could cause error?
In-Reply-To: <DB647D42-81E9-4C68-9B90-74C023C0B160@redhat.com>
References: <309297395.7678675.1483427171321.JavaMail.zimbra@redhat.com>
	<DB647D42-81E9-4C68-9B90-74C023C0B160@redhat.com>
Message-ID: <1320060706.9220253.1483944952056.JavaMail.zimbra@redhat.com>

Thanks Brian.

----- Original Message -----
> From: "Brian Stansberry" <brian.stansberry at redhat.com>
> To: "Bartosz Baranowski" <bbaranow at redhat.com>
> Cc: wildfly-dev at lists.jboss.org
> Sent: Friday, January 6, 2017 5:24:42 PM
> Subject: Re: [wildfly-dev] Add a warning if some configuration change has wider impact and could cause error?
> 
> Any opinions? This is basically a proposal to provide a formal way of
> including warning data in operations responses.
> 
> If we did it it would probably be something like:
> 
> {
>   ?outcome?=>?success?,
>   ?result?=>?foo?,
>   ?response-headers?=> {
>     ?warnings?=[
>         ?blah blah blah?,
>        { ?some_complex_issue: details?>[?x?,?y?,?z?]}
>     ]
>   }
> }
> 
> So a standard response header whose value is a list of ModelNode, different
> messages using different node structures, but the norm being a simple
> string.
> 
> Probably OperationContext would need to add a method addResponseWarning to
> take the details of constructing the header out of the hands of the handler
> authors.
> 
> We?d have to make sure these headers propogate properly through the domain,
> so the effort is more than simply getting standalone.

Most likely. I have very little experience with mgmt ops code in domain mode.

> 
> If the operation is a composite, there are no per-step response headers so
> the warning should not assume that the context of the warning is obvious.
> Warning authors forgetting that point is a possible source of UX issues.

In such case I think either warning has to be very explicit(detailed what/where/why) or it would become 
complex type:
 {
   ?outcome?=>?success?,
   ?result?=>?foo?,
   ?response-headers?=> {
     ?warnings?=> {
        warning => {
           operation => { address,name,parameters}
           message = [""]
           },
        warning => {
           operation => { address,name,parameters}
           message = [""]
           }
     }
   }
 }

or

{
   ?outcome?=>?success?,
   ?result?=>?foo?,
   ?response-headers?=> {
     ?warnings?=> {
        warning => {
           step => ["1"]
           message = [""]
           },
        warning => {
           step => ["2"]
           message = [""]
           }
     }
   }
 }


Though second option might be too cryptic for console users, unless it is well supported.

> 
> Is this worth the effort? This is driven by this one WFCORE-1987 issue, which
> is a bit unusual and in an ideal world would go away. But here?s the general
> pattern involved:
> 
> 2 attributes in separate parts of the model should be consistent or you wil
> get bad runtime behavior
> User can use a CLI batch to set them both atomically, but web console users
> can?t do that, so even if the user will do the right thing there will be a
> moment where they are inconsistent
> So, failing is not good (forces users to use CLI + batch)
> For the same reason a WARN in the log is not great, as unrelated users may be
> required to react to log warns, and will be annoyed by seeing them just
> because their colleague didn?t use a batch
> 
> Is that last statement enough reason to add this functionality vs just using
> the server log to warn, or just not warning at all?

No warning at all is a bad thing. Unless user is very experienced, such UX will be 
perceived as a bad design - well, unless exception/failure will be descriptive enough.
Imagine that you change one tiny detail and server fails to boot up some, seemingly 
unrelated service.

So as is, it seem to be a question of fixing current failure message( which for instance
, in case of WFCORE-1987, does not seem viable) vs standardized warning mechanism. 

> 
> In theory we could use the same facility to report management API deprecation
> warnings back. Doing that might be annoying though. Those are a less serious
> problem than this WFCORE-1987 thing.
> 
> 
> > On Jan 3, 2017, at 1:06 AM, Bartosz Baranowski <bbaranow at redhat.com> wrote:
> > 
> > Hey guys.
> > 
> > 
> > In some cases there are/will be conf changes that might require user to
> > make additional change, for instance [1].
> > In this case, if user does not edit the name in two places, when remote
> > client tries to reach out, it will fail.
> > 
> > As far as I remember, there as of now, there is no formal constraint
> > feature for config values.
> > I've talked with Brian about this and it seems and viable option would be
> > to simply add a header to response.
> > In CLI, it would be clearly visible, in console, it would require a bit  of
> > hacking to present
> > information/hint what needs to be done to configuration to make it work
> > properly.
> > 
> > 
> > Any thoughts?
> > 
> > [1] https://issues.jboss.org/browse/WFCORE-1987
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> --
> Brian Stansberry
> Manager, Senior Principal Software Engineer
> JBoss by Red Hat
> 
> 
> 
>


From bbaranow at redhat.com  Mon Jan  9 02:02:39 2017
From: bbaranow at redhat.com (Bartosz Baranowski)
Date: Mon, 9 Jan 2017 02:02:39 -0500 (EST)
Subject: [wildfly-dev] Add a warning if some configuration change has
 wider impact and could cause error?
In-Reply-To: <E03C1DB7-414F-4D2F-9BB9-F84493D9FC8C@redhat.com>
References: <309297395.7678675.1483427171321.JavaMail.zimbra@redhat.com>
	<DB647D42-81E9-4C68-9B90-74C023C0B160@redhat.com>
	<E03C1DB7-414F-4D2F-9BB9-F84493D9FC8C@redhat.com>
Message-ID: <2034555131.9220905.1483945359022.JavaMail.zimbra@redhat.com>



----- Original Message -----
> From: "Jeff Mesnil" <jmesnil at redhat.com>
> To: "Brian Stansberry" <brian.stansberry at redhat.com>
> Cc: "Bartosz Baranowski" <bbaranow at redhat.com>, wildfly-dev at lists.jboss.org
> Sent: Friday, January 6, 2017 5:48:11 PM
> Subject: Re: [wildfly-dev] Add a warning if some configuration change has wider impact and could cause error?
> 
> 
> > On 6 Jan 2017, at 17:24, Brian Stansberry <brian.stansberry at redhat.com>
> > wrote:
> > 
> > Any opinions? This is basically a proposal to provide a formal way of
> > including warning data in operations responses.
> > 
> > If we did it it would probably be something like:
> > 
> > {
> >  ?outcome?=>?success?,
> >  ?result?=>?foo?,
> >  ?response-headers?=> {
> >    ?warnings?=[
> >        ?blah blah blah?,
> >       { ?some_complex_issue: details?>[?x?,?y?,?z?]}
> >    ]
> >  }
> > }
> 
> We had a similar discussion while implementing :migrate operations for legacy
> subsystems and ended up using a "notification-warnings? parameter in the
> result node (as opposed to the response-headers node).
> 
> > 
> > Is that last statement enough reason to add this functionality vs just
> > using the server log to warn, or just not warning at all?
> 
> It?s nice to have for sure and would help having a consistent reporting
> systems in our management clients (we can imagine having flash warnings in
> the web console and special text in the CLI written to syserr for example).
> 
> In an ideal world, I think that any WARN logged in the server console that is
> caused by a management operation should be reported to the invoker.

Yes, in a way. But difference here is that WARN could come from any place, so we would have to intercept logger.
Rather than rely on dev to be through and be aware of implications of mgmt ops. Just guessing that it might become quite complicated.
For initial implementation I would just stick to original problem.

> I?d go a step further to constrain the warnings model as a dictionary where
> each key is a registered logger ID (e.g. XNIO001007) and a value a i18ned
> string (or a more complex i18ned structure).
> 
> Something like:
> 
> {
>  ?outcome?=>?success?,
>  ?result?=>?foo?,
>  ?response-headers?=> {
>    ?warnings? => {
>        ?XNIO00107? => ?some thing happens?,
>        ?WFCORE1234? => { ?some_complex_issue? => [?x?,?y?,?z?]}
>    }
>  }
> }
> 
> That?d be very helpful to build our knowledge base and report issues.
> 
> > In theory we could use the same facility to report management API
> > deprecation warnings back. Doing that might be annoying though. Those are
> > a less serious problem than this WFCORE-1987 thing.
> 
> If you go that way, you could somehow configure the level of reported
> warnings and let the admin configure the level of reported warnings:
> 
> addResponseWarning(String id, int level, String)
> addResponseWarning(String id, int level, ModelNode data)

Most likely level could be equal to logging.Level.

> 
> We now have a nice new core-management subsystem where the admin could change
> the reported warning level (defaulting to something corresponding to serious
> warnings, as opposed to minor warnings such as API deprecation).

Where? Or are you talking about above example?

> 
> jeff
> 
> --
> Jeff Mesnil
> JBoss, a division of Red Hat
> http://jmesnil.net/
> 
>


From rory.odonnell at oracle.com  Tue Jan 10 05:20:13 2017
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Tue, 10 Jan 2017 10:20:13 +0000
Subject: [wildfly-dev] JDK 9 EA Build 151 is available on java.net
Message-ID: <dcb219ec-6b44-1cf6-da23-1089c95a7233@oracle.com>


Hi Jason/Tomaz,

Best wishes for the New Year.

Dalibor and I will be at FOSDEM '17, Brussels 4 & 5 February. Let us 
know if you will be there, hopefully we can meet up !

*JDK 9 Early Access* b151 <https://jdk9.java.net/download/>  is 
available on java.net

There have been a number of fixes to bugs reported by Open Source 
projects since the last availability email  :

  * JDK-8171377 : Add sun.misc.Unsafe::invokeCleaner
  * JDK-8075793 : Source incompatibility for inference using -source 7
  * JDK-8087303  : LSSerializer pretty print does not work anymore
  * JDK-8167143 :CLDR timezone parsing does not work for all locales

Other changes that maybe of interest:

  * JDK-8066474 : Remove the lib/$ARCH directory from Linux and Solaris
    images
  * JDK-8170428 : Move src.zip to JDK/lib/src.zip

*JEPs intergrated:*

  * JEP 295 <http://openjdk.java.net/jeps/295>: Ahead-of-Time
    Compilation has been integrated in b150.

*Schedule - Milestones since last availability email *

  * *Feature Extension Complete 22nd of December 2016*
  * *Rampdown Started 5th of January 2017
    *
      o Phases in which increasing levels of scrutiny are applied to
        incoming changes.
      o In phase 1, only P1-P3 bugs can be fixed. In phase 2 only
        showstopper bugs can be fixed.

Rgds,Rory

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170110/57241f11/attachment.html 

From tomaz.cerar at gmail.com  Tue Jan 10 05:48:34 2017
From: tomaz.cerar at gmail.com (=?UTF-8?B?VG9tYcW+IENlcmFy?=)
Date: Tue, 10 Jan 2017 11:48:34 +0100
Subject: [wildfly-dev] JDK 9 EA Build 151 is available on java.net
In-Reply-To: <dcb219ec-6b44-1cf6-da23-1089c95a7233@oracle.com>
References: <dcb219ec-6b44-1cf6-da23-1089c95a7233@oracle.com>
Message-ID: <CAMquZP7Fui+ddzw3uLbN11Wg7vw-JT-Hv48dbs_OWDjrQw8W1A@mail.gmail.com>

Hey Rory,

Best wishes for new year also to you and your team.

we already tested b150 and b151 but our testing is mostly blocked by
already fixed https://bugs.openjdk.java.net/browse/JDK-8172158
which I am guessing should be part of upcoming b152.

Beyond that I am cautiously optimistic about our work around JDK9 and
passing our testsuite.

--
tomaz

On Tue, Jan 10, 2017 at 11:20 AM, Rory O'Donnell <rory.odonnell at oracle.com>
wrote:

>
> Hi Jason/Tomaz,
>
> Best wishes for the New Year.
>
> Dalibor and I will be at FOSDEM '17, Brussels 4 & 5 February. Let us know
> if you will be there, hopefully we can meet up !
> *JDK 9 Early Access*  b151 <https://jdk9.java.net/download/>  is
> available on java.net
>
> There have been a number of fixes to bugs reported by Open Source projects
> since the last availability email  :
>
>    - JDK-8171377 : Add sun.misc.Unsafe::invokeCleaner
>    - JDK-8075793 : Source incompatibility for inference using -source 7
>    - JDK-8087303  : LSSerializer pretty print does not work anymore
>    - JDK-8167143 :CLDR timezone parsing does not work for all locales
>
> Other changes that maybe of interest:
>
>    - JDK-8066474 : Remove the lib/$ARCH directory from Linux and Solaris
>    images
>    - JDK-8170428 : Move src.zip to JDK/lib/src.zip
>
> *JEPs intergrated:*
>
>    -  JEP 295 <http://openjdk.java.net/jeps/295>: Ahead-of-Time
>    Compilation has been integrated in b150.
>
> *Schedule - Milestones since last availability email *
>
>    - *Feature Extension Complete 22nd of December 2016*
>    -
> *Rampdown Started 5th of January 2017 *
>       - Phases in which increasing levels of scrutiny are applied to
>       incoming changes.
>       - In phase 1, only P1-P3 bugs can be fixed. In phase 2 only
>       showstopper bugs can be fixed.
>
> Rgds,Rory
>
> --
> Rgds,Rory O'Donnell
> Quality Engineering Manager
> Oracle EMEA , Dublin, Ireland
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170110/999ab200/attachment.html 

From rory.odonnell at oracle.com  Tue Jan 10 06:10:53 2017
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Tue, 10 Jan 2017 11:10:53 +0000
Subject: [wildfly-dev] JDK 9 EA Build 151 is available on java.net
In-Reply-To: <CAMquZP7Fui+ddzw3uLbN11Wg7vw-JT-Hv48dbs_OWDjrQw8W1A@mail.gmail.com>
References: <dcb219ec-6b44-1cf6-da23-1089c95a7233@oracle.com>
	<CAMquZP7Fui+ddzw3uLbN11Wg7vw-JT-Hv48dbs_OWDjrQw8W1A@mail.gmail.com>
Message-ID: <bf20d675-3e48-d51c-8538-75c785859e43@oracle.com>

Thanks for the update Tomaz!

Hopefully meet up at Fosdem ?

Rgds,Rory


On 10/01/2017 10:48, Toma? Cerar wrote:
> Hey Rory,
>
> Best wishes for new year also to you and your team.
>
> we already tested b150 and b151 but our testing is mostly blocked by 
> already fixed https://bugs.openjdk.java.net/browse/JDK-8172158
> which I am guessing should be part of upcoming b152.
>
> Beyond that I am cautiously optimistic about our work around JDK9 and 
> passing our testsuite.
>
> --
> tomaz
>
> On Tue, Jan 10, 2017 at 11:20 AM, Rory O'Donnell 
> <rory.odonnell at oracle.com <mailto:rory.odonnell at oracle.com>> wrote:
>
>
>     Hi Jason/Tomaz,
>
>     Best wishes for the New Year.
>
>     Dalibor and I will be at FOSDEM '17, Brussels 4 & 5 February. Let
>     us know if you will be there, hopefully we can meet up !
>
>     *JDK 9 Early Access* b151 <https://jdk9.java.net/download/> is
>     available on java.net <http://java.net>
>
>     There have been a number of fixes to bugs reported by Open Source
>     projects since the last availability email  :
>
>       * JDK-8171377 : Add sun.misc.Unsafe::invokeCleaner
>       * JDK-8075793 : Source incompatibility for inference using
>         -source 7
>       * JDK-8087303  : LSSerializer pretty print does not work anymore
>       * JDK-8167143 :CLDR timezone parsing does not work for all locales
>
>     Other changes that maybe of interest:
>
>       * JDK-8066474 : Remove the lib/$ARCH directory from Linux and
>         Solaris images
>       * JDK-8170428 : Move src.zip to JDK/lib/src.zip
>
>     *JEPs intergrated:*
>
>       * JEP 295 <http://openjdk.java.net/jeps/295>: Ahead-of-Time
>         Compilation has been integrated in b150.
>
>     *Schedule - Milestones since last availability email *
>
>       * *Feature Extension Complete 22nd of December 2016*
>       * *Rampdown Started 5th of January 2017
>         *
>           o Phases in which increasing levels of scrutiny are applied
>             to incoming changes.
>           o In phase 1, only P1-P3 bugs can be fixed. In phase 2 only
>             showstopper bugs can be fixed.
>
>     Rgds,Rory
>
>     -- 
>     Rgds,Rory O'Donnell
>     Quality Engineering Manager
>     Oracle EMEA , Dublin, Ireland
>
>

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170110/1db82925/attachment-0001.html 

From rareddy at redhat.com  Wed Jan 11 09:34:15 2017
From: rareddy at redhat.com (Ramesh Reddy)
Date: Wed, 11 Jan 2017 09:34:15 -0500 (EST)
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <814364143.5474880.1484145082498.JavaMail.zimbra@redhat.com>
Message-ID: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>

Hi,

For Teiid project (http://teiid.org), we typically deploy a .xml or .VDB (zip archive) file to define a virtual database artifact. We are planning to deliver a feature where a virtual database is written in DDL, for this we would like to deploy a file artifact like "foo-vdb.ddl".

I have written deployment processors for it, and added DEPLOYMENT_ROOT mounter to recognize the deployment artifact etc, however during the deployment scanning, WildFly always looks at anything other than ".xml" file as zip archive, or a exploded zip archive, so that it can do VFS mount on that file. I would like to add this ".ddl" extension file exactly similar to ".xml" file. Is there any way to achieve this?

Thank you.

Ramesh..

From brian.stansberry at redhat.com  Wed Jan 11 11:36:14 2017
From: brian.stansberry at redhat.com (Brian Stansberry)
Date: Wed, 11 Jan 2017 10:36:14 -0600
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
References: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
Message-ID: <967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>

I?ll defer to our deployment processing gurus re the mount question.

As an aside though, for this to work with the deployment-scanner subsystem we?ll need to add some logic. Right now it would just ignore the file.

> On Jan 11, 2017, at 8:34 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
> 
> Hi,
> 
> For Teiid project (http://teiid.org), we typically deploy a .xml or .VDB (zip archive) file to define a virtual database artifact. We are planning to deliver a feature where a virtual database is written in DDL, for this we would like to deploy a file artifact like "foo-vdb.ddl".
> 
> I have written deployment processors for it, and added DEPLOYMENT_ROOT mounter to recognize the deployment artifact etc, however during the deployment scanning, WildFly always looks at anything other than ".xml" file as zip archive, or a exploded zip archive, so that it can do VFS mount on that file. I would like to add this ".ddl" extension file exactly similar to ".xml" file. Is there any way to achieve this?
> 
> Thank you.
> 
> Ramesh..
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat





From rareddy at redhat.com  Wed Jan 11 14:51:59 2017
From: rareddy at redhat.com (Ramesh Reddy)
Date: Wed, 11 Jan 2017 14:51:59 -0500 (EST)
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>
References: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
	<967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>
Message-ID: <654796948.5475761.1484164319256.JavaMail.zimbra@redhat.com>

It would be good if the logic can be changed such that any thing other than with extension jar,war,ear and zip to consider them as file based deployments rather than zip archive based deployments.

----- Original Message -----
> I?ll defer to our deployment processing gurus re the mount question.
> 
> As an aside though, for this to work with the deployment-scanner subsystem
> we?ll need to add some logic. Right now it would just ignore the file.
> 
> > On Jan 11, 2017, at 8:34 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
> > 
> > Hi,
> > 
> > For Teiid project (http://teiid.org), we typically deploy a .xml or .VDB
> > (zip archive) file to define a virtual database artifact. We are planning
> > to deliver a feature where a virtual database is written in DDL, for this
> > we would like to deploy a file artifact like "foo-vdb.ddl".
> > 
> > I have written deployment processors for it, and added DEPLOYMENT_ROOT
> > mounter to recognize the deployment artifact etc, however during the
> > deployment scanning, WildFly always looks at anything other than ".xml"
> > file as zip archive, or a exploded zip archive, so that it can do VFS
> > mount on that file. I would like to add this ".ddl" extension file exactly
> > similar to ".xml" file. Is there any way to achieve this?
> > 
> > Thank you.
> > 
> > Ramesh..
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> --
> Brian Stansberry
> Manager, Senior Principal Software Engineer
> JBoss by Red Hat
> 
> 
> 
> 
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


From stuart.w.douglas at gmail.com  Wed Jan 11 15:29:40 2017
From: stuart.w.douglas at gmail.com (Stuart Douglas)
Date: Thu, 12 Jan 2017 07:29:40 +1100
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <654796948.5475761.1484164319256.JavaMail.zimbra@redhat.com>
References: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
	<967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>
	<654796948.5475761.1484164319256.JavaMail.zimbra@redhat.com>
Message-ID: <CAAoo=c4ZK1bXzPpGP_Ff9YR3m4HJMYJnNWaO7ZR1b9wXrbdBiw@mail.gmail.com>

The problem with that is that the majority of the time this would be a
mistake on the users part (at the moment it is 100% of the time, as we only
deploy stuff we know how to process).

I think that ideally this should be pluggable. It would be easy enough to
make DeploymentRootMountProcessor handle different file types (for instance
add an AttachmentList of known REAL file types, and then install a DUP
before it to add .ddl to the list).

Using capabilities and requirements we could probably add something similar
to the deployment scanner as well, basically your subsystem would look for
the deployment scanner capability, and if it is installed we could have
some API to add additional file types (although this may be a bit racey, as
the service may not know all file types for the initial scan, which could
give odd results in some circumstances. There is probably a solution to
this but I can't think of one off the top of my head).

Stuart



On Thu, Jan 12, 2017 at 6:51 AM, Ramesh Reddy <rareddy at redhat.com> wrote:

> It would be good if the logic can be changed such that any thing other
> than with extension jar,war,ear and zip to consider them as file based
> deployments rather than zip archive based deployments.
>
> ----- Original Message -----
> > I?ll defer to our deployment processing gurus re the mount question.
> >
> > As an aside though, for this to work with the deployment-scanner
> subsystem
> > we?ll need to add some logic. Right now it would just ignore the file.
> >
> > > On Jan 11, 2017, at 8:34 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
> > >
> > > Hi,
> > >
> > > For Teiid project (http://teiid.org), we typically deploy a .xml or
> .VDB
> > > (zip archive) file to define a virtual database artifact. We are
> planning
> > > to deliver a feature where a virtual database is written in DDL, for
> this
> > > we would like to deploy a file artifact like "foo-vdb.ddl".
> > >
> > > I have written deployment processors for it, and added DEPLOYMENT_ROOT
> > > mounter to recognize the deployment artifact etc, however during the
> > > deployment scanning, WildFly always looks at anything other than ".xml"
> > > file as zip archive, or a exploded zip archive, so that it can do VFS
> > > mount on that file. I would like to add this ".ddl" extension file
> exactly
> > > similar to ".xml" file. Is there any way to achieve this?
> > >
> > > Thank you.
> > >
> > > Ramesh..
> > > _______________________________________________
> > > wildfly-dev mailing list
> > > wildfly-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> >
> > --
> > Brian Stansberry
> > Manager, Senior Principal Software Engineer
> > JBoss by Red Hat
> >
> >
> >
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170112/461eb438/attachment.html 

From bmcwhirt at redhat.com  Wed Jan 11 16:09:08 2017
From: bmcwhirt at redhat.com (Bob McWhirter)
Date: Wed, 11 Jan 2017 21:09:08 +0000
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <CAAoo=c4ZK1bXzPpGP_Ff9YR3m4HJMYJnNWaO7ZR1b9wXrbdBiw@mail.gmail.com>
References: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
	<967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>
	<654796948.5475761.1484164319256.JavaMail.zimbra@redhat.com>
	<CAAoo=c4ZK1bXzPpGP_Ff9YR3m4HJMYJnNWaO7ZR1b9wXrbdBiw@mail.gmail.com>
Message-ID: <CA+45JvELTtVvfa9dqnJ3-QZiFahb4OJ2pWCs9FmxyV0VN9=JVA@mail.gmail.com>

TorqueBox allows deployment of .yaml files. It's pluggable enough.

Bob

On Wed, Jan 11, 2017 at 3:32 PM Stuart Douglas <stuart.w.douglas at gmail.com>
wrote:

> The problem with that is that the majority of the time this would be a
> mistake on the users part (at the moment it is 100% of the time, as we only
> deploy stuff we know how to process).
>
> I think that ideally this should be pluggable. It would be easy enough to
> make DeploymentRootMountProcessor handle different file types (for instance
> add an AttachmentList of known REAL file types, and then install a DUP
> before it to add .ddl to the list).
>
> Using capabilities and requirements we could probably add something
> similar to the deployment scanner as well, basically your subsystem would
> look for the deployment scanner capability, and if it is installed we could
> have some API to add additional file types (although this may be a bit
> racey, as the service may not know all file types for the initial scan,
> which could give odd results in some circumstances. There is probably a
> solution to this but I can't think of one off the top of my head).
>
> Stuart
>
>
>
> On Thu, Jan 12, 2017 at 6:51 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
>
> It would be good if the logic can be changed such that any thing other
> than with extension jar,war,ear and zip to consider them as file based
> deployments rather than zip archive based deployments.
>
>
>
>
>
> ----- Original Message -----
>
>
> > I?ll defer to our deployment processing gurus re the mount question.
>
>
> >
>
>
> > As an aside though, for this to work with the deployment-scanner
> subsystem
>
>
> > we?ll need to add some logic. Right now it would just ignore the file.
>
>
> >
>
>
> > > On Jan 11, 2017, at 8:34 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
>
>
> > >
>
>
> > > Hi,
>
>
> > >
>
>
> > > For Teiid project (http://teiid.org), we typically deploy a .xml or
> .VDB
>
>
> > > (zip archive) file to define a virtual database artifact. We are
> planning
>
>
> > > to deliver a feature where a virtual database is written in DDL, for
> this
>
>
> > > we would like to deploy a file artifact like "foo-vdb.ddl".
>
>
> > >
>
>
> > > I have written deployment processors for it, and added DEPLOYMENT_ROOT
>
>
> > > mounter to recognize the deployment artifact etc, however during the
>
>
> > > deployment scanning, WildFly always looks at anything other than ".xml"
>
>
> > > file as zip archive, or a exploded zip archive, so that it can do VFS
>
>
> > > mount on that file. I would like to add this ".ddl" extension file
> exactly
>
>
> > > similar to ".xml" file. Is there any way to achieve this?
>
>
> > >
>
>
> > > Thank you.
>
>
> > >
>
>
> > > Ramesh..
>
>
> > > _______________________________________________
>
>
> > > wildfly-dev mailing list
>
>
> > > wildfly-dev at lists.jboss.org
>
>
> > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
> >
>
>
> > --
>
>
> > Brian Stansberry
>
>
> > Manager, Senior Principal Software Engineer
>
>
> > JBoss by Red Hat
>
>
> >
>
>
> >
>
>
> >
>
>
> >
>
>
> > _______________________________________________
>
>
> > wildfly-dev mailing list
>
>
> > wildfly-dev at lists.jboss.org
>
>
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
>
>
>
> _______________________________________________
>
>
> wildfly-dev mailing list
>
>
> wildfly-dev at lists.jboss.org
>
>
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
>
>
> _______________________________________________
>
> wildfly-dev mailing list
>
> wildfly-dev at lists.jboss.org
>
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170111/e9ad6b64/attachment-0001.html 

From stuart.w.douglas at gmail.com  Wed Jan 11 16:38:03 2017
From: stuart.w.douglas at gmail.com (Stuart Douglas)
Date: Thu, 12 Jan 2017 08:38:03 +1100
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <CA+45JvELTtVvfa9dqnJ3-QZiFahb4OJ2pWCs9FmxyV0VN9=JVA@mail.gmail.com>
References: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
	<967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>
	<654796948.5475761.1484164319256.JavaMail.zimbra@redhat.com>
	<CAAoo=c4ZK1bXzPpGP_Ff9YR3m4HJMYJnNWaO7ZR1b9wXrbdBiw@mail.gmail.com>
	<CA+45JvELTtVvfa9dqnJ3-QZiFahb4OJ2pWCs9FmxyV0VN9=JVA@mail.gmail.com>
Message-ID: <CAAoo=c5TouVfaRRT4iW08o+UL5Ht5Jv9oOfXmghYwES_F9FhuQ@mail.gmail.com>

Ah, I guess you could just install an earlier DUP that attaches the
DEPLOYMENT_ROOT yourself, then DeploymentRootMountProcessor will not run.

I'm not sure about the deployment scanner though. I think if you have a
.dodeploy file it can attempt to deploy anything, however I can't see any
way to make it deploy arbitrary files without a marker file.

Stuart

On Thu, Jan 12, 2017 at 8:09 AM, Bob McWhirter <bmcwhirt at redhat.com> wrote:

> TorqueBox allows deployment of .yaml files. It's pluggable enough.
>
> Bob
>
> On Wed, Jan 11, 2017 at 3:32 PM Stuart Douglas <stuart.w.douglas at gmail.com>
> wrote:
>
>> The problem with that is that the majority of the time this would be a
>> mistake on the users part (at the moment it is 100% of the time, as we only
>> deploy stuff we know how to process).
>>
>> I think that ideally this should be pluggable. It would be easy enough to
>> make DeploymentRootMountProcessor handle different file types (for instance
>> add an AttachmentList of known REAL file types, and then install a DUP
>> before it to add .ddl to the list).
>>
>> Using capabilities and requirements we could probably add something
>> similar to the deployment scanner as well, basically your subsystem would
>> look for the deployment scanner capability, and if it is installed we could
>> have some API to add additional file types (although this may be a bit
>> racey, as the service may not know all file types for the initial scan,
>> which could give odd results in some circumstances. There is probably a
>> solution to this but I can't think of one off the top of my head).
>>
>> Stuart
>>
>>
>>
>> On Thu, Jan 12, 2017 at 6:51 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
>>
>> It would be good if the logic can be changed such that any thing other
>> than with extension jar,war,ear and zip to consider them as file based
>> deployments rather than zip archive based deployments.
>>
>>
>>
>>
>>
>> ----- Original Message -----
>>
>>
>> > I?ll defer to our deployment processing gurus re the mount question.
>>
>>
>> >
>>
>>
>> > As an aside though, for this to work with the deployment-scanner
>> subsystem
>>
>>
>> > we?ll need to add some logic. Right now it would just ignore the file.
>>
>>
>> >
>>
>>
>> > > On Jan 11, 2017, at 8:34 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
>>
>>
>> > >
>>
>>
>> > > Hi,
>>
>>
>> > >
>>
>>
>> > > For Teiid project (http://teiid.org), we typically deploy a .xml or
>> .VDB
>>
>>
>> > > (zip archive) file to define a virtual database artifact. We are
>> planning
>>
>>
>> > > to deliver a feature where a virtual database is written in DDL, for
>> this
>>
>>
>> > > we would like to deploy a file artifact like "foo-vdb.ddl".
>>
>>
>> > >
>>
>>
>> > > I have written deployment processors for it, and added DEPLOYMENT_ROOT
>>
>>
>> > > mounter to recognize the deployment artifact etc, however during the
>>
>>
>> > > deployment scanning, WildFly always looks at anything other than
>> ".xml"
>>
>>
>> > > file as zip archive, or a exploded zip archive, so that it can do VFS
>>
>>
>> > > mount on that file. I would like to add this ".ddl" extension file
>> exactly
>>
>>
>> > > similar to ".xml" file. Is there any way to achieve this?
>>
>>
>> > >
>>
>>
>> > > Thank you.
>>
>>
>> > >
>>
>>
>> > > Ramesh..
>>
>>
>> > > _______________________________________________
>>
>>
>> > > wildfly-dev mailing list
>>
>>
>> > > wildfly-dev at lists.jboss.org
>>
>>
>> > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>>
>> >
>>
>>
>> > --
>>
>>
>> > Brian Stansberry
>>
>>
>> > Manager, Senior Principal Software Engineer
>>
>>
>> > JBoss by Red Hat
>>
>>
>> >
>>
>>
>> >
>>
>>
>> >
>>
>>
>> >
>>
>>
>> > _______________________________________________
>>
>>
>> > wildfly-dev mailing list
>>
>>
>> > wildfly-dev at lists.jboss.org
>>
>>
>> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>>
>>
>>
>>
>> _______________________________________________
>>
>>
>> wildfly-dev mailing list
>>
>>
>> wildfly-dev at lists.jboss.org
>>
>>
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>>
>>
>>
>> _______________________________________________
>>
>> wildfly-dev mailing list
>>
>> wildfly-dev at lists.jboss.org
>>
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170112/29459815/attachment.html 

From brian.stansberry at redhat.com  Wed Jan 11 18:25:49 2017
From: brian.stansberry at redhat.com (Brian Stansberry)
Date: Wed, 11 Jan 2017 17:25:49 -0600
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <CAAoo=c5TouVfaRRT4iW08o+UL5Ht5Jv9oOfXmghYwES_F9FhuQ@mail.gmail.com>
References: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
	<967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>
	<654796948.5475761.1484164319256.JavaMail.zimbra@redhat.com>
	<CAAoo=c4ZK1bXzPpGP_Ff9YR3m4HJMYJnNWaO7ZR1b9wXrbdBiw@mail.gmail.com>
	<CA+45JvELTtVvfa9dqnJ3-QZiFahb4OJ2pWCs9FmxyV0VN9=JVA@mail.gmail.com>
	<CAAoo=c5TouVfaRRT4iW08o+UL5Ht5Jv9oOfXmghYwES_F9FhuQ@mail.gmail.com>
Message-ID: <32568D44-CD92-4F90-9CBC-76E5D7F19B30@redhat.com>


> On Jan 11, 2017, at 3:38 PM, Stuart Douglas <stuart.w.douglas at gmail.com> wrote:
> 
> Ah, I guess you could just install an earlier DUP that attaches the DEPLOYMENT_ROOT yourself, then DeploymentRootMountProcessor will not run.
> 
> I'm not sure about the deployment scanner though. I think if you have a .dodeploy file it can attempt to deploy anything, however I can't see any way to make it deploy arbitrary files without a marker file.
> 

Perhaps that is for the best. I was thinking about doing the capability driven thing you mentioned so people can register other types. But the primary point of registering other types would be to register some ?file is complete? check handler that could be used to confirm that the file is complete. Without this deploying without a marker is not safe as the scanner can read a partially copied file. I?m not sure any ?file is complete? check handler is practical for other file types, and if not just using the marker is the correct behavior. 

Perhaps we could just count on java.nio.channels.FileLock for that, but that?s documented as only being advisory and working with a locked file has other limitations.[1]


[1] https://docs.oracle.com/javase/8/docs/api/java/nio/channels/FileLock.html#pdep

> Stuart
> 
> On Thu, Jan 12, 2017 at 8:09 AM, Bob McWhirter <bmcwhirt at redhat.com> wrote:
> TorqueBox allows deployment of .yaml files. It's pluggable enough. 
> 
> Bob
> 
> On Wed, Jan 11, 2017 at 3:32 PM Stuart Douglas <stuart.w.douglas at gmail.com> wrote:
> The problem with that is that the majority of the time this would be a mistake on the users part (at the moment it is 100% of the time, as we only deploy stuff we know how to process).
> 
> I think that ideally this should be pluggable. It would be easy enough to make DeploymentRootMountProcessor handle different file types (for instance add an AttachmentList of known REAL file types, and then install a DUP before it to add .ddl to the list).
> 
> Using capabilities and requirements we could probably add something similar to the deployment scanner as well, basically your subsystem would look for the deployment scanner capability, and if it is installed we could have some API to add additional file types (although this may be a bit racey, as the service may not know all file types for the initial scan, which could give odd results in some circumstances. There is probably a solution to this but I can't think of one off the top of my head).
> 
> Stuart
> 
> 
> 
> On Thu, Jan 12, 2017 at 6:51 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
> It would be good if the logic can be changed such that any thing other than with extension jar,war,ear and zip to consider them as file based deployments rather than zip archive based deployments.
> 
> 
> 
> 
> 
> ----- Original Message -----
> 
> 
> > I?ll defer to our deployment processing gurus re the mount question.
> 
> 
> >
> 
> 
> > As an aside though, for this to work with the deployment-scanner subsystem
> 
> 
> > we?ll need to add some logic. Right now it would just ignore the file.
> 
> 
> >
> 
> 
> > > On Jan 11, 2017, at 8:34 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
> 
> 
> > >
> 
> 
> > > Hi,
> 
> 
> > >
> 
> 
> > > For Teiid project (http://teiid.org), we typically deploy a .xml or .VDB
> 
> 
> > > (zip archive) file to define a virtual database artifact. We are planning
> 
> 
> > > to deliver a feature where a virtual database is written in DDL, for this
> 
> 
> > > we would like to deploy a file artifact like "foo-vdb.ddl".
> 
> 
> > >
> 
> 
> > > I have written deployment processors for it, and added DEPLOYMENT_ROOT
> 
> 
> > > mounter to recognize the deployment artifact etc, however during the
> 
> 
> > > deployment scanning, WildFly always looks at anything other than ".xml"
> 
> 
> > > file as zip archive, or a exploded zip archive, so that it can do VFS
> 
> 
> > > mount on that file. I would like to add this ".ddl" extension file exactly
> 
> 
> > > similar to ".xml" file. Is there any way to achieve this?
> 
> 
> > >
> 
> 
> > > Thank you.
> 
> 
> > >
> 
> 
> > > Ramesh..
> 
> 
> > > _______________________________________________
> 
> 
> > > wildfly-dev mailing list
> 
> 
> > > wildfly-dev at lists.jboss.org
> 
> 
> > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> 
> >
> 
> 
> > --
> 
> 
> > Brian Stansberry
> 
> 
> > Manager, Senior Principal Software Engineer
> 
> 
> > JBoss by Red Hat
> 
> 
> >
> 
> 
> >
> 
> 
> >
> 
> 
> >
> 
> 
> > _______________________________________________
> 
> 
> > wildfly-dev mailing list
> 
> 
> > wildfly-dev at lists.jboss.org
> 
> 
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> 
> 
> 
> 
> _______________________________________________
> 
> 
> wildfly-dev mailing list
> 
> 
> wildfly-dev at lists.jboss.org
> 
> 
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> 
> 
> _______________________________________________
> 
> wildfly-dev mailing list
> 
> wildfly-dev at lists.jboss.org
> 
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat





From ppalaga at redhat.com  Thu Jan 12 03:53:59 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Thu, 12 Jan 2017 09:53:59 +0100
Subject: [wildfly-dev] Maven 3.3.1+ coming to wildfly and wildfly-core
In-Reply-To: <b10e0573-d1bc-e631-a48e-1318bf7dcb60@redhat.com>
References: <6d206f5a-a45a-6bcb-1deb-2f4feee34ba2@redhat.com>
	<CADGwLpi_rGjn9rCSLMVy6YtD2Z1zs2jcjd8LpAoXE=FFY59nMg@mail.gmail.com>
	<f55703ff-7a5b-4bf9-79bf-54936efb253b@redhat.com>
	<CAJtdNN6wgL7xzGOEB+xmBpzj2x85KEQJ_M+ojxU6HNL3JqQGOQ@mail.gmail.com>
	<b10e0573-d1bc-e631-a48e-1318bf7dcb60@redhat.com>
Message-ID: <6ce4cc4c-2661-7901-4fd1-9f96dd14029a@redhat.com>

The PR https://github.com/wildfly/wildfly-core/pull/2017 got merged in 
WF Core (thanks Brian!) and I am going to prepare a similar PR for WF. -- P

On 2016-12-14 09:36, Peter Palaga wrote:
> Hi Scott, OK, let's keep the build.(sh|bat) scripts for now. I'll change
> them (1) to wrap mvnw and (2) to print a warning that they are
> deprecated in favor of mvnw. -- P
>
> On 2016-12-14 01:04, Scott Marlow wrote:
>> Hi Peter,
>>
>> On Tue, Dec 13, 2016 at 5:26 PM, Peter Palaga <ppalaga at redhat.com> wrote:
>>> Hi James,
>>>
>>> On 2016-12-13 22:51, James Perkins wrote:
>>>> Hi Peter,
>>>> It may not be a big deal because it's just for builds, but instead of
>>>> deleting the build.(sh|bat) maybe we just make those invoke the
>>>> appropriate mvnw command instead.
>>>
>>> That would be easy to do, but can you name a situation where it would
>>> actually matter?
>>
>> We have a few (compliance) test scripts that invoke build.sh to build
>> the app server from source.  I'm not sure how widespread use of
>> build.sh is in other test environments but we probably could ask that
>> question via a separate topic.
>>
>> Scott
>>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


From rareddy at redhat.com  Thu Jan 12 10:30:01 2017
From: rareddy at redhat.com (Ramesh Reddy)
Date: Thu, 12 Jan 2017 10:30:01 -0500 (EST)
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <CA+45JvELTtVvfa9dqnJ3-QZiFahb4OJ2pWCs9FmxyV0VN9=JVA@mail.gmail.com>
References: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
	<967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>
	<654796948.5475761.1484164319256.JavaMail.zimbra@redhat.com>
	<CAAoo=c4ZK1bXzPpGP_Ff9YR3m4HJMYJnNWaO7ZR1b9wXrbdBiw@mail.gmail.com>
	<CA+45JvELTtVvfa9dqnJ3-QZiFahb4OJ2pWCs9FmxyV0VN9=JVA@mail.gmail.com>
Message-ID: <1922895198.5478841.1484235001611.JavaMail.zimbra@redhat.com>

Bob, 

Can you please point me to your code references for this? 

Ramesh.. 

----- Original Message -----

> TorqueBox allows deployment of .yaml files. It's pluggable enough.

> Bob

> On Wed, Jan 11, 2017 at 3:32 PM Stuart Douglas < stuart.w.douglas at gmail.com >
> wrote:

> > The problem with that is that the majority of the time this would be a
> > mistake on the users part (at the moment it is 100% of the time, as we only
> > deploy stuff we know how to process).
> 

> > I think that ideally this should be pluggable. It would be easy enough to
> > make DeploymentRootMountProcessor handle different file types (for instance
> > add an AttachmentList of known REAL file types, and then install a DUP
> > before it to add .ddl to the list).
> 

> > Using capabilities and requirements we could probably add something similar
> > to the deployment scanner as well, basically your subsystem would look for
> > the deployment scanner capability, and if it is installed we could have
> > some
> > API to add additional file types (although this may be a bit racey, as the
> > service may not know all file types for the initial scan, which could give
> > odd results in some circumstances. There is probably a solution to this but
> > I can't think of one off the top of my head).
> 

> > Stuart
> 

> > On Thu, Jan 12, 2017 at 6:51 AM, Ramesh Reddy < rareddy at redhat.com > wrote:
> 

> > > It would be good if the logic can be changed such that any thing other
> > > than
> > > with extension jar,war,ear and zip to consider them as file based
> > > deployments rather than zip archive based deployments.
> > 
> 

> > > ----- Original Message -----
> > 
> 

> > > > I?ll defer to our deployment processing gurus re the mount question.
> > 
> 

> > > >
> > 
> 

> > > > As an aside though, for this to work with the deployment-scanner
> > > > subsystem
> > 
> 

> > > > we?ll need to add some logic. Right now it would just ignore the file.
> > 
> 

> > > >
> > 
> 

> > > > > On Jan 11, 2017, at 8:34 AM, Ramesh Reddy < rareddy at redhat.com >
> > > > > wrote:
> > 
> 

> > > > >
> > 
> 

> > > > > Hi,
> > 
> 

> > > > >
> > 
> 

> > > > > For Teiid project ( http://teiid.org ), we typically deploy a .xml or
> > > > > .VDB
> > 
> 

> > > > > (zip archive) file to define a virtual database artifact. We are
> > > > > planning
> > 
> 

> > > > > to deliver a feature where a virtual database is written in DDL, for
> > > > > this
> > 
> 

> > > > > we would like to deploy a file artifact like "foo-vdb.ddl".
> > 
> 

> > > > >
> > 
> 

> > > > > I have written deployment processors for it, and added
> > > > > DEPLOYMENT_ROOT
> > 
> 

> > > > > mounter to recognize the deployment artifact etc, however during the
> > 
> 

> > > > > deployment scanning, WildFly always looks at anything other than
> > > > > ".xml"
> > 
> 

> > > > > file as zip archive, or a exploded zip archive, so that it can do VFS
> > 
> 

> > > > > mount on that file. I would like to add this ".ddl" extension file
> > > > > exactly
> > 
> 

> > > > > similar to ".xml" file. Is there any way to achieve this?
> > 
> 

> > > > >
> > 
> 

> > > > > Thank you.
> > 
> 

> > > > >
> > 
> 

> > > > > Ramesh..
> > 
> 

> > > > > _______________________________________________
> > 
> 

> > > > > wildfly-dev mailing list
> > 
> 

> > > > > wildfly-dev at lists.jboss.org
> > 
> 

> > > > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> > 
> 

> > > >
> > 
> 

> > > > --
> > 
> 

> > > > Brian Stansberry
> > 
> 

> > > > Manager, Senior Principal Software Engineer
> > 
> 

> > > > JBoss by Red Hat
> > 
> 

> > > >
> > 
> 

> > > >
> > 
> 

> > > >
> > 
> 

> > > >
> > 
> 

> > > > _______________________________________________
> > 
> 

> > > > wildfly-dev mailing list
> > 
> 

> > > > wildfly-dev at lists.jboss.org
> > 
> 

> > > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> > 
> 

> > > _______________________________________________
> > 
> 

> > > wildfly-dev mailing list
> > 
> 

> > > wildfly-dev at lists.jboss.org
> > 
> 

> > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> > 
> 

> > _______________________________________________
> 

> > wildfly-dev mailing list
> 

> > wildfly-dev at lists.jboss.org
> 

> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170112/2e3e3fd0/attachment-0001.html 

From anmiller at redhat.com  Thu Jan 12 10:43:03 2017
From: anmiller at redhat.com (Andrig Miller)
Date: Thu, 12 Jan 2017 08:43:03 -0700
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <32568D44-CD92-4F90-9CBC-76E5D7F19B30@redhat.com>
References: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
	<967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>
	<654796948.5475761.1484164319256.JavaMail.zimbra@redhat.com>
	<CAAoo=c4ZK1bXzPpGP_Ff9YR3m4HJMYJnNWaO7ZR1b9wXrbdBiw@mail.gmail.com>
	<CA+45JvELTtVvfa9dqnJ3-QZiFahb4OJ2pWCs9FmxyV0VN9=JVA@mail.gmail.com>
	<CAAoo=c5TouVfaRRT4iW08o+UL5Ht5Jv9oOfXmghYwES_F9FhuQ@mail.gmail.com>
	<32568D44-CD92-4F90-9CBC-76E5D7F19B30@redhat.com>
Message-ID: <CAF1-sp7hyp4ra+1A1OUHscif2az2KV5O6pdregOhJ6X+cS9xRQ@mail.gmail.com>

On Wed, Jan 11, 2017 at 4:25 PM, Brian Stansberry <
brian.stansberry at redhat.com> wrote:

>
> > On Jan 11, 2017, at 3:38 PM, Stuart Douglas <stuart.w.douglas at gmail.com>
> wrote:
> >
> > Ah, I guess you could just install an earlier DUP that attaches the
> DEPLOYMENT_ROOT yourself, then DeploymentRootMountProcessor will not run.
> >
> > I'm not sure about the deployment scanner though. I think if you have a
> .dodeploy file it can attempt to deploy anything, however I can't see any
> way to make it deploy arbitrary files without a marker file.
> >
>
> Perhaps that is for the best. I was thinking about doing the capability
> driven thing you mentioned so people can register other types. But the
> primary point of registering other types would be to register some ?file is
> complete? check handler that could be used to confirm that the file is
> complete. Without this deploying without a marker is not safe as the
> scanner can read a partially copied file. I?m not sure any ?file is
> complete? check handler is practical for other file types, and if not just
> using the marker is the correct behavior.
>
> Perhaps we could just count on java.nio.channels.FileLock for that, but
> that?s documented as only being advisory and working with a locked file has
> other limitations.[1]
>
>
> [1] https://docs.oracle.com/javase/8/docs/api/java/nio/
> channels/FileLock.html#pdep


?This is advisory, because it depends on a specific POSIX API on LInux/Unix
systems, that may or may not be there depending on the underlying file
system.  We ran into this with HornetQ, since it relies on this for its
journal lock for HA.  The GFS 2 file system didn't have this lock?

?so HA for HornetQ didn't work.  We had to implement a native code
workaround to use the BSD based API that was functionally the same as the
POSIX API.  Eventually the GFS team did add the POSIX API.  It turned out,
all the other file systems that have been tested all worked and had the
underlying POSIX API support (including NFS).

Andy?

>
>
> > Stuart
> >
> > On Thu, Jan 12, 2017 at 8:09 AM, Bob McWhirter <bmcwhirt at redhat.com>
> wrote:
> > TorqueBox allows deployment of .yaml files. It's pluggable enough.
> >
> > Bob
> >
> > On Wed, Jan 11, 2017 at 3:32 PM Stuart Douglas <
> stuart.w.douglas at gmail.com> wrote:
> > The problem with that is that the majority of the time this would be a
> mistake on the users part (at the moment it is 100% of the time, as we only
> deploy stuff we know how to process).
> >
> > I think that ideally this should be pluggable. It would be easy enough
> to make DeploymentRootMountProcessor handle different file types (for
> instance add an AttachmentList of known REAL file types, and then install a
> DUP before it to add .ddl to the list).
> >
> > Using capabilities and requirements we could probably add something
> similar to the deployment scanner as well, basically your subsystem would
> look for the deployment scanner capability, and if it is installed we could
> have some API to add additional file types (although this may be a bit
> racey, as the service may not know all file types for the initial scan,
> which could give odd results in some circumstances. There is probably a
> solution to this but I can't think of one off the top of my head).
> >
> > Stuart
> >
> >
> >
> > On Thu, Jan 12, 2017 at 6:51 AM, Ramesh Reddy <rareddy at redhat.com>
> wrote:
> > It would be good if the logic can be changed such that any thing other
> than with extension jar,war,ear and zip to consider them as file based
> deployments rather than zip archive based deployments.
> >
> >
> >
> >
> >
> > ----- Original Message -----
> >
> >
> > > I?ll defer to our deployment processing gurus re the mount question.
> >
> >
> > >
> >
> >
> > > As an aside though, for this to work with the deployment-scanner
> subsystem
> >
> >
> > > we?ll need to add some logic. Right now it would just ignore the file.
> >
> >
> > >
> >
> >
> > > > On Jan 11, 2017, at 8:34 AM, Ramesh Reddy <rareddy at redhat.com>
> wrote:
> >
> >
> > > >
> >
> >
> > > > Hi,
> >
> >
> > > >
> >
> >
> > > > For Teiid project (http://teiid.org), we typically deploy a .xml or
> .VDB
> >
> >
> > > > (zip archive) file to define a virtual database artifact. We are
> planning
> >
> >
> > > > to deliver a feature where a virtual database is written in DDL, for
> this
> >
> >
> > > > we would like to deploy a file artifact like "foo-vdb.ddl".
> >
> >
> > > >
> >
> >
> > > > I have written deployment processors for it, and added
> DEPLOYMENT_ROOT
> >
> >
> > > > mounter to recognize the deployment artifact etc, however during the
> >
> >
> > > > deployment scanning, WildFly always looks at anything other than
> ".xml"
> >
> >
> > > > file as zip archive, or a exploded zip archive, so that it can do VFS
> >
> >
> > > > mount on that file. I would like to add this ".ddl" extension file
> exactly
> >
> >
> > > > similar to ".xml" file. Is there any way to achieve this?
> >
> >
> > > >
> >
> >
> > > > Thank you.
> >
> >
> > > >
> >
> >
> > > > Ramesh..
> >
> >
> > > > _______________________________________________
> >
> >
> > > > wildfly-dev mailing list
> >
> >
> > > > wildfly-dev at lists.jboss.org
> >
> >
> > > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> >
> >
> > >
> >
> >
> > > --
> >
> >
> > > Brian Stansberry
> >
> >
> > > Manager, Senior Principal Software Engineer
> >
> >
> > > JBoss by Red Hat
> >
> >
> > >
> >
> >
> > >
> >
> >
> > >
> >
> >
> > >
> >
> >
> > > _______________________________________________
> >
> >
> > > wildfly-dev mailing list
> >
> >
> > > wildfly-dev at lists.jboss.org
> >
> >
> > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> >
> >
> >
> >
> >
> > _______________________________________________
> >
> >
> > wildfly-dev mailing list
> >
> >
> > wildfly-dev at lists.jboss.org
> >
> >
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> >
> >
> >
> > _______________________________________________
> >
> > wildfly-dev mailing list
> >
> > wildfly-dev at lists.jboss.org
> >
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > wildfly-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
> --
> Brian Stansberry
> Manager, Senior Principal Software Engineer
> JBoss by Red Hat
>
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>



-- 
Andrig (Andy) T. Miller
Global Platform Director, Middleware
Red Hat, Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170112/c3956af9/attachment.html 

From sanne at hibernate.org  Fri Jan 13 07:03:52 2017
From: sanne at hibernate.org (Sanne Grinovero)
Date: Fri, 13 Jan 2017 12:03:52 +0000
Subject: [wildfly-dev] JPADependencyProcessor "infecting" classpath with the
	wrong Javassist version
Message-ID: <CAFm4XO1kY=X7_B0AGe=S0eWEgQVF=79mf3-uHXTcFRXiTmkr_A@mail.gmail.com>

Hi all,

Scott sent a nice PR to Wildfly a while back to fix the problem:
 - https://github.com/wildfly/wildfly/pull/9305

It wasn't merged, I guess it's not a priority for WildFly but let me
clarify that without such fixes it's impossible for people to use
newer versions of Hibernate ORM on WildFly, and I suspect lots of pain
as well for other libraries using Javassist.

There's quite some people in the Hibernate community who expressed
interest in using a not-so-stale version as the one which is typically
available in the latest stable release of WildFly.

To make this happen, all Hibernate projects are now publishing
"WildFly modules" which can be easily downloaded as additional drop-in
layer.
Granted these are not for everyone but we get good feedback from the
power users interested, and not least this allows us to develop all
our projects while regularly testing integration with WildFly, making
sure that the eventual integration goes smoother.

The current problem is that the WildFly JPADependencyProcessor adds
the wrong version of javassist to the deployments, and there's no way
for us to prevent this or override this, making it impossible to use a
recent version of Hibernate ORM as it requires a newer version.
 - https://github.com/wildfly/wildfly/blob/6b61a6003f704221f66dcd9f418bcb7af88fb9ab/jpa/subsystem/src/main/java/org/jboss/as/jpa/processor/JPADependencyProcessor.java#L95

We'd highly appreciate if that PR could be merged? Including on
product branches please, as enforcing a dependency which is neither
needed nor desired is going to break a long list of other frameworks
as well.

Thanks,
Sanne

From smarlow at redhat.com  Fri Jan 13 09:46:58 2017
From: smarlow at redhat.com (Scott Marlow)
Date: Fri, 13 Jan 2017 09:46:58 -0500
Subject: [wildfly-dev] JPADependencyProcessor "infecting" classpath with
 the wrong Javassist version
In-Reply-To: <CAFm4XO1kY=X7_B0AGe=S0eWEgQVF=79mf3-uHXTcFRXiTmkr_A@mail.gmail.com>
References: <CAFm4XO1kY=X7_B0AGe=S0eWEgQVF=79mf3-uHXTcFRXiTmkr_A@mail.gmail.com>
Message-ID: <CAJtdNN7_bq_DapLJ_pnHBDfo+QYbgYxa=X=fAqqjUxKCTptSUQ@mail.gmail.com>

Is that the change you want merge or is it the
https://github.com/wildfly/wildfly/pull/8474 change which updates
JPADependencyProcessor to not export Javassist jars to the application
classloader?



On Fri, Jan 13, 2017 at 7:03 AM, Sanne Grinovero <sanne at hibernate.org> wrote:
> Hi all,
>
> Scott sent a nice PR to Wildfly a while back to fix the problem:
>  - https://github.com/wildfly/wildfly/pull/9305
>
> It wasn't merged, I guess it's not a priority for WildFly but let me
> clarify that without such fixes it's impossible for people to use
> newer versions of Hibernate ORM on WildFly, and I suspect lots of pain
> as well for other libraries using Javassist.
>
> There's quite some people in the Hibernate community who expressed
> interest in using a not-so-stale version as the one which is typically
> available in the latest stable release of WildFly.
>
> To make this happen, all Hibernate projects are now publishing
> "WildFly modules" which can be easily downloaded as additional drop-in
> layer.
> Granted these are not for everyone but we get good feedback from the
> power users interested, and not least this allows us to develop all
> our projects while regularly testing integration with WildFly, making
> sure that the eventual integration goes smoother.
>
> The current problem is that the WildFly JPADependencyProcessor adds
> the wrong version of javassist to the deployments, and there's no way
> for us to prevent this or override this, making it impossible to use a
> recent version of Hibernate ORM as it requires a newer version.
>  - https://github.com/wildfly/wildfly/blob/6b61a6003f704221f66dcd9f418bcb7af88fb9ab/jpa/subsystem/src/main/java/org/jboss/as/jpa/processor/JPADependencyProcessor.java#L95
>
> We'd highly appreciate if that PR could be merged? Including on
> product branches please, as enforcing a dependency which is neither
> needed nor desired is going to break a long list of other frameworks
> as well.
>
> Thanks,
> Sanne
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

From hpehl at redhat.com  Fri Jan 13 09:52:04 2017
From: hpehl at redhat.com (Harald Pehl)
Date: Fri, 13 Jan 2017 15:52:04 +0100
Subject: [wildfly-dev] Next version of the management console (HAL.next)
Message-ID: <CAPZNLwzjSJE7kPCJ=ttOHKxeG17KydqMM3cUxZ-=hGVw6HX-Eg@mail.gmail.com>

We're currently working on the next version of the management console
(HAL.next [1]). HAL.next continues with the column based navigation
which was introduced in WildFly 9. It's also still written in Java and
based on GWT. However most parts of the console have been rewritten
for several reasons:

- Make use of new features in Java 8 and GWT 2.8 and prepare for GWT 3.0
- Introduce new features which were not possible with the old
architecture (keyboard navigation, bookmarks, macro recording, ...)
- Adopt PatternFly [2]
- Cleanup codebase (remove deprecated and unused code)

For more details about the motivation and the complete list of new
features see the README at [1].

Development is still ongoing and we plan to replace the existing
console with HAL.next in WildFly 11 earliest. We have now reached a
milestone where it makes sense to get feedback from a bigger audience.
That's why we like to invite everyone to take a look at the new
console. Suggestions, ideas, comments and bug reports are welcome.
Please use GitHub issues [3] to file issues and discuss enhancements.

To get started, please follow the README [4]. The easiest way is
probably to use the docker image at [5]. It contains the latest
WildFly 11 build with HAL.next as management console.


[1] https://github.com/hal/hal.next
[2] https://www.patternfly.org/
[3] https://github.com/hal/hal.next/issues
[4] https://github.com/hal/hal.next#running
[5] https://hub.docker.com/r/hpehl/hal-next/

-- 
Harald Pehl
JBoss by Red Hat
http://hpehl.info

From bmcwhirt at redhat.com  Fri Jan 13 10:17:15 2017
From: bmcwhirt at redhat.com (Bob McWhirter)
Date: Fri, 13 Jan 2017 10:17:15 -0500
Subject: [wildfly-dev] Deployment Root type other than XML or ZIP
In-Reply-To: <1922895198.5478841.1484235001611.JavaMail.zimbra@redhat.com>
References: <278499244.5474896.1484145255514.JavaMail.zimbra@redhat.com>
	<967843F3-9FF4-4EB7-9F16-28110FC613DA@redhat.com>
	<654796948.5475761.1484164319256.JavaMail.zimbra@redhat.com>
	<CAAoo=c4ZK1bXzPpGP_Ff9YR3m4HJMYJnNWaO7ZR1b9wXrbdBiw@mail.gmail.com>
	<CA+45JvELTtVvfa9dqnJ3-QZiFahb4OJ2pWCs9FmxyV0VN9=JVA@mail.gmail.com>
	<1922895198.5478841.1484235001611.JavaMail.zimbra@redhat.com>
Message-ID: <CA+45JvG=MKd60s51m0gHdVbYrPgBtGe3KvXOgcL9N9kANdA98w@mail.gmail.com>

Here?s what TorqueBox used to use:

https://github.com/projectodd/jboss-polyglot/blob/master/modules/core/src/main/java/org/projectodd/polyglot/core/processors/DescriptorRootMountProcessor.java

On Thu, Jan 12, 2017 at 10:30 AM, Ramesh Reddy <rareddy at redhat.com> wrote:

> Bob,
>
> Can you please point me to your code references for this?
>
> Ramesh..
>
> ------------------------------
>
> TorqueBox allows deployment of .yaml files. It's pluggable enough.
>
> Bob
>
> On Wed, Jan 11, 2017 at 3:32 PM Stuart Douglas <stuart.w.douglas at gmail.com>
> wrote:
>
>> The problem with that is that the majority of the time this would be a
>> mistake on the users part (at the moment it is 100% of the time, as we only
>> deploy stuff we know how to process).
>>
>> I think that ideally this should be pluggable. It would be easy enough to
>> make DeploymentRootMountProcessor handle different file types (for instance
>> add an AttachmentList of known REAL file types, and then install a DUP
>> before it to add .ddl to the list).
>>
>> Using capabilities and requirements we could probably add something
>> similar to the deployment scanner as well, basically your subsystem would
>> look for the deployment scanner capability, and if it is installed we could
>> have some API to add additional file types (although this may be a bit
>> racey, as the service may not know all file types for the initial scan,
>> which could give odd results in some circumstances. There is probably a
>> solution to this but I can't think of one off the top of my head).
>>
>> Stuart
>>
>>
>>
>> On Thu, Jan 12, 2017 at 6:51 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
>>
>> It would be good if the logic can be changed such that any thing other
>> than with extension jar,war,ear and zip to consider them as file based
>> deployments rather than zip archive based deployments.
>>
>>
>>
>>
>>
>> ----- Original Message -----
>>
>>
>> > I?ll defer to our deployment processing gurus re the mount question.
>>
>>
>> >
>>
>>
>> > As an aside though, for this to work with the deployment-scanner
>> subsystem
>>
>>
>> > we?ll need to add some logic. Right now it would just ignore the file.
>>
>>
>> >
>>
>>
>> > > On Jan 11, 2017, at 8:34 AM, Ramesh Reddy <rareddy at redhat.com> wrote:
>>
>>
>> > >
>>
>>
>> > > Hi,
>>
>>
>> > >
>>
>>
>> > > For Teiid project (http://teiid.org), we typically deploy a .xml or
>> .VDB
>>
>>
>> > > (zip archive) file to define a virtual database artifact. We are
>> planning
>>
>>
>> > > to deliver a feature where a virtual database is written in DDL, for
>> this
>>
>>
>> > > we would like to deploy a file artifact like "foo-vdb.ddl".
>>
>>
>> > >
>>
>>
>> > > I have written deployment processors for it, and added DEPLOYMENT_ROOT
>>
>>
>> > > mounter to recognize the deployment artifact etc, however during the
>>
>>
>> > > deployment scanning, WildFly always looks at anything other than
>> ".xml"
>>
>>
>> > > file as zip archive, or a exploded zip archive, so that it can do VFS
>>
>>
>> > > mount on that file. I would like to add this ".ddl" extension file
>> exactly
>>
>>
>> > > similar to ".xml" file. Is there any way to achieve this?
>>
>>
>> > >
>>
>>
>> > > Thank you.
>>
>>
>> > >
>>
>>
>> > > Ramesh..
>>
>>
>> > > _______________________________________________
>>
>>
>> > > wildfly-dev mailing list
>>
>>
>> > > wildfly-dev at lists.jboss.org
>>
>>
>> > > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>>
>> >
>>
>>
>> > --
>>
>>
>> > Brian Stansberry
>>
>>
>> > Manager, Senior Principal Software Engineer
>>
>>
>> > JBoss by Red Hat
>>
>>
>> >
>>
>>
>> >
>>
>>
>> >
>>
>>
>> >
>>
>>
>> > _______________________________________________
>>
>>
>> > wildfly-dev mailing list
>>
>>
>> > wildfly-dev at lists.jboss.org
>>
>>
>> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>>
>>
>>
>>
>> _______________________________________________
>>
>>
>> wildfly-dev mailing list
>>
>>
>> wildfly-dev at lists.jboss.org
>>
>>
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>>
>>
>>
>> _______________________________________________
>>
>> wildfly-dev mailing list
>>
>> wildfly-dev at lists.jboss.org
>>
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170113/16ae79a2/attachment.html 

From sanne at hibernate.org  Fri Jan 13 11:02:44 2017
From: sanne at hibernate.org (Sanne Grinovero)
Date: Fri, 13 Jan 2017 16:02:44 +0000
Subject: [wildfly-dev] JPADependencyProcessor "infecting" classpath with
 the wrong Javassist version
In-Reply-To: <CAJtdNN7_bq_DapLJ_pnHBDfo+QYbgYxa=X=fAqqjUxKCTptSUQ@mail.gmail.com>
References: <CAFm4XO1kY=X7_B0AGe=S0eWEgQVF=79mf3-uHXTcFRXiTmkr_A@mail.gmail.com>
	<CAJtdNN7_bq_DapLJ_pnHBDfo+QYbgYxa=X=fAqqjUxKCTptSUQ@mail.gmail.com>
Message-ID: <CAFm4XO0PuW75AS7YLx_hXs0pJZQpbjVW84niQGaSxKtyntOJOw@mail.gmail.com>

Hi Scott,
you're right, both would be very welcome but removing Javassist is the
most critical one.

Thanks,
Sanne

On 13 January 2017 at 14:46, Scott Marlow <smarlow at redhat.com> wrote:
> Is that the change you want merge or is it the
> https://github.com/wildfly/wildfly/pull/8474 change which updates
> JPADependencyProcessor to not export Javassist jars to the application
> classloader?
>
>
>
> On Fri, Jan 13, 2017 at 7:03 AM, Sanne Grinovero <sanne at hibernate.org> wrote:
>> Hi all,
>>
>> Scott sent a nice PR to Wildfly a while back to fix the problem:
>>  - https://github.com/wildfly/wildfly/pull/9305
>>
>> It wasn't merged, I guess it's not a priority for WildFly but let me
>> clarify that without such fixes it's impossible for people to use
>> newer versions of Hibernate ORM on WildFly, and I suspect lots of pain
>> as well for other libraries using Javassist.
>>
>> There's quite some people in the Hibernate community who expressed
>> interest in using a not-so-stale version as the one which is typically
>> available in the latest stable release of WildFly.
>>
>> To make this happen, all Hibernate projects are now publishing
>> "WildFly modules" which can be easily downloaded as additional drop-in
>> layer.
>> Granted these are not for everyone but we get good feedback from the
>> power users interested, and not least this allows us to develop all
>> our projects while regularly testing integration with WildFly, making
>> sure that the eventual integration goes smoother.
>>
>> The current problem is that the WildFly JPADependencyProcessor adds
>> the wrong version of javassist to the deployments, and there's no way
>> for us to prevent this or override this, making it impossible to use a
>> recent version of Hibernate ORM as it requires a newer version.
>>  - https://github.com/wildfly/wildfly/blob/6b61a6003f704221f66dcd9f418bcb7af88fb9ab/jpa/subsystem/src/main/java/org/jboss/as/jpa/processor/JPADependencyProcessor.java#L95
>>
>> We'd highly appreciate if that PR could be merged? Including on
>> product branches please, as enforcing a dependency which is neither
>> needed nor desired is going to break a long list of other frameworks
>> as well.
>>
>> Thanks,
>> Sanne
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev

From sanne at hibernate.org  Fri Jan 13 11:57:45 2017
From: sanne at hibernate.org (Sanne Grinovero)
Date: Fri, 13 Jan 2017 16:57:45 +0000
Subject: [wildfly-dev] Subsystem Inclusion Policy & Role of Feature
 Packs & Add-ons
In-Reply-To: <2F07456A-0D5C-4084-B4F2-F818372ECD03@redhat.com>
References: <2F07456A-0D5C-4084-B4F2-F818372ECD03@redhat.com>
Message-ID: <CAFm4XO0P59CYd+Qfz5YOmXPOap5i6kKvYpEA4HdQ+D=_QoQp5g@mail.gmail.com>

Hi Jason, all,

allow me to resuscitate this thread after a year and a half.

These directions have been very clear and useful, I was needing to
look them up again today. Wondering: would you give the same
directions today?

Especially considering WildFly Swarm, should we still avoid making
Hibernate feature packs / fractions / patches ?

We would like to make sure that Hibernate users (ORM, OGM, Search,
...) could somewhat easily access our latest versions as soon as they
are released, without having to wait for several other "downstream"
projects to incorporate our latest.

Thanks,
Sanne

On 3 August 2015 at 17:37, Jason Greene <jason.greene at redhat.com> wrote:
> Hello Everyone,
>
> Recently there has been some confusion about how subsystems should be distributed, and whether or not they should be part of the WildFly repository.
>
> There are three primary use-cases for distributing a subsystem.
>
> #1 - Inclusion in an official WildFly distribution
> #2 - A user installable "add-on distribution" which can be dropped on top of a WildFly Distribution (see [A])
> #3 - A separate, independant, customized distribution, with a differing identity. Possibly build but not required as a layer (see [A])
>
> If you are after #1, then the subsystem source code (defined as the portion of code which integrates with the server using the subsystem facilities) MUST be included in the WildFly repo. This is because subsystems heavily impact the stability of the server and our compliance with our strict management compatibility policy, and additionally it allows for us to keep all included subsystems up to date with core infrastructure changes such as capabilities and requirements, and the upcoming elytron security integration. Under this approach, a feature-pack is unlikely to be used, as it would likely just be part of the full feature-pack. It could very well be that we would introduce a different more expansive feature-pack in the future defining a larger distribution foot-print, however, there are currently no plans to do so.
>
> If you are after #2, then you do not want a feature-pack, as feature-packs are just for building custom server distributions. If your use-case is #2 you are by definition not a custom server distribution, but rather a set of modules built the normal maven way.
>
> If you are after #3, then you likely wish to use the feature-pack mechanism to make it easy to produce your custom distribution. This facility would allow you to keep your source repository limited to just the new subsystems you introduce, and pull the rest of the server bits via a maven dep. It is important, that you change the identity of the server (see [A]), such that patches for the official WildFly server are not accidentally installed.
>
> Thanks!
>
> [A] https://developer.jboss.org/wiki/LayeredDistributionsAndModulePathOrganization
>
>
>
>
>
>
>
> --
> Jason T. Greene
> WildFly Lead / JBoss EAP Platform Architect
> JBoss, a division of Red Hat
>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


From stuart.w.douglas at gmail.com  Sun Jan 15 17:23:03 2017
From: stuart.w.douglas at gmail.com (Stuart Douglas)
Date: Mon, 16 Jan 2017 09:23:03 +1100
Subject: [wildfly-dev] JPADependencyProcessor "infecting" classpath with
 the wrong Javassist version
In-Reply-To: <CAFm4XO1kY=X7_B0AGe=S0eWEgQVF=79mf3-uHXTcFRXiTmkr_A@mail.gmail.com>
References: <CAFm4XO1kY=X7_B0AGe=S0eWEgQVF=79mf3-uHXTcFRXiTmkr_A@mail.gmail.com>
Message-ID: <CAAoo=c4z=+w1WMinJZSSpZS1MLYQpYfv8BfYw6yV0ecbRp=zmA@mail.gmail.com>

Is there any reason why we can't update to the latest version of Javassist?
I know this does not prevent similar issues in future, but we should
probably be using the most recent version if there is no reason not to.

My issue with https://github.com/wildfly/wildfly/pull/8474 is that in
general we should not be using export="true", as it makes it impossible to
exclude the dependency using jboss-deployment-structure.xml. As a result I
don't really see how this PR is an improvement, as it goes from the
javassist deployment being added directly to the deployment to it being
added indirectly via an export.

Stuart

On Fri, Jan 13, 2017 at 11:03 PM, Sanne Grinovero <sanne at hibernate.org>
wrote:

> Hi all,
>
> Scott sent a nice PR to Wildfly a while back to fix the problem:
>  - https://github.com/wildfly/wildfly/pull/9305
>
> It wasn't merged, I guess it's not a priority for WildFly but let me
> clarify that without such fixes it's impossible for people to use
> newer versions of Hibernate ORM on WildFly, and I suspect lots of pain
> as well for other libraries using Javassist.
>
> There's quite some people in the Hibernate community who expressed
> interest in using a not-so-stale version as the one which is typically
> available in the latest stable release of WildFly.
>
> To make this happen, all Hibernate projects are now publishing
> "WildFly modules" which can be easily downloaded as additional drop-in
> layer.
> Granted these are not for everyone but we get good feedback from the
> power users interested, and not least this allows us to develop all
> our projects while regularly testing integration with WildFly, making
> sure that the eventual integration goes smoother.
>
> The current problem is that the WildFly JPADependencyProcessor adds
> the wrong version of javassist to the deployments, and there's no way
> for us to prevent this or override this, making it impossible to use a
> recent version of Hibernate ORM as it requires a newer version.
>  - https://github.com/wildfly/wildfly/blob/6b61a6003f704221f66dcd9f418bcb
> 7af88fb9ab/jpa/subsystem/src/main/java/org/jboss/as/jpa/processor/
> JPADependencyProcessor.java#L95
>
> We'd highly appreciate if that PR could be merged? Including on
> product branches please, as enforcing a dependency which is neither
> needed nor desired is going to break a long list of other frameworks
> as well.
>
> Thanks,
> Sanne
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170116/298869dd/attachment.html 

From hbraun at redhat.com  Mon Jan 16 04:52:40 2017
From: hbraun at redhat.com (Heiko Braun)
Date: Mon, 16 Jan 2017 10:52:40 +0100
Subject: [wildfly-dev] Next version of the management console (HAL.next)
In-Reply-To: <CAPZNLwzjSJE7kPCJ=ttOHKxeG17KydqMM3cUxZ-=hGVw6HX-Eg@mail.gmail.com>
References: <CAPZNLwzjSJE7kPCJ=ttOHKxeG17KydqMM3cUxZ-=hGVw6HX-Eg@mail.gmail.com>
Message-ID: <571156F8-3A44-4370-8D3E-35B1B3033D2A@redhat.com>



I know it?s really challenging to do a major refactoring to support the next major version of GWT, while retaining the critical business logic and UX behind the console. That said, I am really happy to see the team managed to keep the user flows and functional requirements, but elevating the base project kernel to a new and much improved version. Excellent work Harald and the team!

Regards, Heiko

> On 13 Jan 2017, at 15:52, Harald Pehl <hpehl at redhat.com> wrote:
> 
> We're currently working on the next version of the management console
> (HAL.next [1]). HAL.next continues with the column based navigation
> which was introduced in WildFly 9. It's also still written in Java and
> based on GWT. However most parts of the console have been rewritten
> for several reasons:
> 
> - Make use of new features in Java 8 and GWT 2.8 and prepare for GWT 3.0
> - Introduce new features which were not possible with the old
> architecture (keyboard navigation, bookmarks, macro recording, ...)
> - Adopt PatternFly [2]
> - Cleanup codebase (remove deprecated and unused code)
> 
> For more details about the motivation and the complete list of new
> features see the README at [1].
> 
> Development is still ongoing and we plan to replace the existing
> console with HAL.next in WildFly 11 earliest. We have now reached a
> milestone where it makes sense to get feedback from a bigger audience.
> That's why we like to invite everyone to take a look at the new
> console. Suggestions, ideas, comments and bug reports are welcome.
> Please use GitHub issues [3] to file issues and discuss enhancements.
> 
> To get started, please follow the README [4]. The easiest way is
> probably to use the docker image at [5]. It contains the latest
> WildFly 11 build with HAL.next as management console.
> 
> 
> [1] https://github.com/hal/hal.next
> [2] https://www.patternfly.org/
> [3] https://github.com/hal/hal.next/issues
> [4] https://github.com/hal/hal.next#running
> [5] https://hub.docker.com/r/hpehl/hal-next/
> 
> -- 
> Harald Pehl
> JBoss by Red Hat
> http://hpehl.info
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev



From sanne at hibernate.org  Mon Jan 16 06:13:12 2017
From: sanne at hibernate.org (Sanne Grinovero)
Date: Mon, 16 Jan 2017 11:13:12 +0000
Subject: [wildfly-dev] JPADependencyProcessor "infecting" classpath with
 the wrong Javassist version
In-Reply-To: <CAAoo=c4z=+w1WMinJZSSpZS1MLYQpYfv8BfYw6yV0ecbRp=zmA@mail.gmail.com>
References: <CAFm4XO1kY=X7_B0AGe=S0eWEgQVF=79mf3-uHXTcFRXiTmkr_A@mail.gmail.com>
	<CAAoo=c4z=+w1WMinJZSSpZS1MLYQpYfv8BfYw6yV0ecbRp=zmA@mail.gmail.com>
Message-ID: <CAFm4XO3WXSws8MX7qL2CUs+isdDS6usk4YHMQDDBwtBtWWjfCw@mail.gmail.com>

Hi Stuart,

On 15 January 2017 at 22:23, Stuart Douglas <stuart.w.douglas at gmail.com> wrote:
> Is there any reason why we can't update to the latest version of Javassist?

Sure we can update Javassist as well, as long as we pick a version
which is compatible with the (slightly old) version of Hibernate ORM
included in WildFly.
This might require a backport of the Javassist upgrade to the older
stable branch of Hibernate ORM and a new release though, so I'd treat
it as an orthogonal issue especially as such a plan would require its
own set of approvers.

> I know this does not prevent similar issues in future, but we should
> probably be using the most recent version if there is no reason not to.

Right, but bear with me the "not prevent similar issues in future" is
crucial to my problem here. We need to regularly test next-gen
Hibernate ORM while we work on it: we want to have ci.hibernate.org
run the WildFly integration tests on each commit of Hibernate ORM, so
upgrading Javassist - or any other dependency - is an event which
might happen multiple times a week and we don't want to wait for a
WildFly release each time this is needed.
It's really about controlling our own dependencies, to keep
development quick and practical.

> My issue with https://github.com/wildfly/wildfly/pull/8474 is that in
> general we should not be using export="true", as it makes it impossible to
> exclude the dependency using jboss-deployment-structure.xml. As a result I
> don't really see how this PR is an improvement, as it goes from the
> javassist deployment being added directly to the deployment to it being
> added indirectly via an export.

Interesting point, I hadn't thought about users of
jboss-deployment-structure.xml. But still, anyone making use of
`org.hibernate:main` (as it is today) will strictly require to have
`org.javassist:main` on their classpath, so one will not want to
exclude it; while people using say `org.hibernate:5.2.6.Final` will
definitely not want `org.javassist:main` on their classpath as it will
cause subtle problems. In short, it's not a grey area but it's very
clear when it's required or not and we can automate this decision.
So I think the modules definitions of each `org.hibernate:xyz` module
should point to the right instance of the javassist module, rather
than force the users to craft an appropriate structure xml file to
amend for our bad choices?

I guess it is currently important for people to be able to exclude
`org.javassist` because it's being injected aggressively, for example
your apps get "infected" by javassist even if you want to use a
different JPA implementation which has no business at all with
Javassist.
On the other hand, I don't think we can support Hibernate users who
simply disable the right version of javassist using the deployment
structure, so if someone really wants to exclude org.javassist:main I
would suggest them to edit the module file of ORM, or better yet use
their own (and self-supported) ORM module.

By moving it to the static definition of the module, at least I have
the option to use my own set of Hibernate ORM modules and choose an
updated Javassist slot.

Hibernate ORM also has the option to use Byte Buddy now as an
alternative to Javassist; when using the new alternative enhancer one
might also want to get rid of the Javassist dependency altogether,
which power users can achieve by editing their own module, or using
the "next-gen" modules we regularly provide with each Hibernate
release. Fixing the JPADependencyProcessor would make it easy for
people to use newer modules by just layering the new edition.

Thanks,
Sanne


>
> Stuart
>
> On Fri, Jan 13, 2017 at 11:03 PM, Sanne Grinovero <sanne at hibernate.org>
> wrote:
>>
>> Hi all,
>>
>> Scott sent a nice PR to Wildfly a while back to fix the problem:
>>  - https://github.com/wildfly/wildfly/pull/9305
>>
>> It wasn't merged, I guess it's not a priority for WildFly but let me
>> clarify that without such fixes it's impossible for people to use
>> newer versions of Hibernate ORM on WildFly, and I suspect lots of pain
>> as well for other libraries using Javassist.
>>
>> There's quite some people in the Hibernate community who expressed
>> interest in using a not-so-stale version as the one which is typically
>> available in the latest stable release of WildFly.
>>
>> To make this happen, all Hibernate projects are now publishing
>> "WildFly modules" which can be easily downloaded as additional drop-in
>> layer.
>> Granted these are not for everyone but we get good feedback from the
>> power users interested, and not least this allows us to develop all
>> our projects while regularly testing integration with WildFly, making
>> sure that the eventual integration goes smoother.
>>
>> The current problem is that the WildFly JPADependencyProcessor adds
>> the wrong version of javassist to the deployments, and there's no way
>> for us to prevent this or override this, making it impossible to use a
>> recent version of Hibernate ORM as it requires a newer version.
>>  -
>> https://github.com/wildfly/wildfly/blob/6b61a6003f704221f66dcd9f418bcb7af88fb9ab/jpa/subsystem/src/main/java/org/jboss/as/jpa/processor/JPADependencyProcessor.java#L95
>>
>> We'd highly appreciate if that PR could be merged? Including on
>> product branches please, as enforcing a dependency which is neither
>> needed nor desired is going to break a long list of other frameworks
>> as well.
>>
>> Thanks,
>> Sanne
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>

From brian.stansberry at redhat.com  Mon Jan 16 11:26:00 2017
From: brian.stansberry at redhat.com (Brian Stansberry)
Date: Mon, 16 Jan 2017 10:26:00 -0600
Subject: [wildfly-dev] Next version of the management console (HAL.next)
In-Reply-To: <571156F8-3A44-4370-8D3E-35B1B3033D2A@redhat.com>
References: <CAPZNLwzjSJE7kPCJ=ttOHKxeG17KydqMM3cUxZ-=hGVw6HX-Eg@mail.gmail.com>
	<571156F8-3A44-4370-8D3E-35B1B3033D2A@redhat.com>
Message-ID: <CB22411B-8F1F-456D-AACF-6B8D292D1FAE@redhat.com>

+10. You guys are doing great.

> On Jan 16, 2017, at 3:52 AM, Heiko Braun <hbraun at redhat.com> wrote:
> 
> 
> 
> I know it?s really challenging to do a major refactoring to support the next major version of GWT, while retaining the critical business logic and UX behind the console. That said, I am really happy to see the team managed to keep the user flows and functional requirements, but elevating the base project kernel to a new and much improved version. Excellent work Harald and the team!
> 
> Regards, Heiko
> 
>> On 13 Jan 2017, at 15:52, Harald Pehl <hpehl at redhat.com> wrote:
>> 
>> We're currently working on the next version of the management console
>> (HAL.next [1]). HAL.next continues with the column based navigation
>> which was introduced in WildFly 9. It's also still written in Java and
>> based on GWT. However most parts of the console have been rewritten
>> for several reasons:
>> 
>> - Make use of new features in Java 8 and GWT 2.8 and prepare for GWT 3.0
>> - Introduce new features which were not possible with the old
>> architecture (keyboard navigation, bookmarks, macro recording, ...)
>> - Adopt PatternFly [2]
>> - Cleanup codebase (remove deprecated and unused code)
>> 
>> For more details about the motivation and the complete list of new
>> features see the README at [1].
>> 
>> Development is still ongoing and we plan to replace the existing
>> console with HAL.next in WildFly 11 earliest. We have now reached a
>> milestone where it makes sense to get feedback from a bigger audience.
>> That's why we like to invite everyone to take a look at the new
>> console. Suggestions, ideas, comments and bug reports are welcome.
>> Please use GitHub issues [3] to file issues and discuss enhancements.
>> 
>> To get started, please follow the README [4]. The easiest way is
>> probably to use the docker image at [5]. It contains the latest
>> WildFly 11 build with HAL.next as management console.
>> 
>> 
>> [1] https://github.com/hal/hal.next
>> [2] https://www.patternfly.org/
>> [3] https://github.com/hal/hal.next/issues
>> [4] https://github.com/hal/hal.next#running
>> [5] https://hub.docker.com/r/hpehl/hal-next/
>> 
>> -- 
>> Harald Pehl
>> JBoss by Red Hat
>> http://hpehl.info
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> 
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat





From rory.odonnell at oracle.com  Wed Jan 18 04:54:38 2017
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Wed, 18 Jan 2017 09:54:38 +0000
Subject: [wildfly-dev] JDK 9 EA Build 151 is available on java.net
In-Reply-To: <CAMquZP7Fui+ddzw3uLbN11Wg7vw-JT-Hv48dbs_OWDjrQw8W1A@mail.gmail.com>
References: <dcb219ec-6b44-1cf6-da23-1089c95a7233@oracle.com>
	<CAMquZP7Fui+ddzw3uLbN11Wg7vw-JT-Hv48dbs_OWDjrQw8W1A@mail.gmail.com>
Message-ID: <e7ac4c6c-88d9-92d2-32fd-dff0371ef087@oracle.com>

Hi Tomaz,

The fix is in b152, available now,  let us know if that works ?

Rgds,Rory


On 10/01/2017 10:48, Toma? Cerar wrote:
> Hey Rory,
>
> Best wishes for new year also to you and your team.
>
> we already tested b150 and b151 but our testing is mostly blocked by 
> already fixed https://bugs.openjdk.java.net/browse/JDK-8172158
> which I am guessing should be part of upcoming b152.
>
> Beyond that I am cautiously optimistic about our work around JDK9 and 
> passing our testsuite.
>
> --
> tomaz
>
> On Tue, Jan 10, 2017 at 11:20 AM, Rory O'Donnell 
> <rory.odonnell at oracle.com <mailto:rory.odonnell at oracle.com>> wrote:
>
>
>     Hi Jason/Tomaz,
>
>     Best wishes for the New Year.
>
>     Dalibor and I will be at FOSDEM '17, Brussels 4 & 5 February. Let
>     us know if you will be there, hopefully we can meet up !
>
>     *JDK 9 Early Access* b151 <https://jdk9.java.net/download/> is
>     available on java.net <http://java.net>
>
>     There have been a number of fixes to bugs reported by Open Source
>     projects since the last availability email  :
>
>       * JDK-8171377 : Add sun.misc.Unsafe::invokeCleaner
>       * JDK-8075793 : Source incompatibility for inference using
>         -source 7
>       * JDK-8087303  : LSSerializer pretty print does not work anymore
>       * JDK-8167143 :CLDR timezone parsing does not work for all locales
>
>     Other changes that maybe of interest:
>
>       * JDK-8066474 : Remove the lib/$ARCH directory from Linux and
>         Solaris images
>       * JDK-8170428 : Move src.zip to JDK/lib/src.zip
>
>     *JEPs intergrated:*
>
>       * JEP 295 <http://openjdk.java.net/jeps/295>: Ahead-of-Time
>         Compilation has been integrated in b150.
>
>     *Schedule - Milestones since last availability email *
>
>       * *Feature Extension Complete 22nd of December 2016*
>       * *Rampdown Started 5th of January 2017
>         *
>           o Phases in which increasing levels of scrutiny are applied
>             to incoming changes.
>           o In phase 1, only P1-P3 bugs can be fixed. In phase 2 only
>             showstopper bugs can be fixed.
>
>     Rgds,Rory
>
>     -- 
>     Rgds,Rory O'Donnell
>     Quality Engineering Manager
>     Oracle EMEA , Dublin, Ireland
>
>

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170118/2246d227/attachment-0001.html 

From tomaz.cerar at gmail.com  Wed Jan 18 08:25:43 2017
From: tomaz.cerar at gmail.com (=?UTF-8?B?VG9tYcW+IENlcmFy?=)
Date: Wed, 18 Jan 2017 14:25:43 +0100
Subject: [wildfly-dev] JDK 9 EA Build 151 is available on java.net
In-Reply-To: <e7ac4c6c-88d9-92d2-32fd-dff0371ef087@oracle.com>
References: <dcb219ec-6b44-1cf6-da23-1089c95a7233@oracle.com>
	<CAMquZP7Fui+ddzw3uLbN11Wg7vw-JT-Hv48dbs_OWDjrQw8W1A@mail.gmail.com>
	<e7ac4c6c-88d9-92d2-32fd-dff0371ef087@oracle.com>
Message-ID: <CAMquZP6pdQMvdovJ1Ra0DCMHiJqiXEbpmhr9kAw7eXULw33PuA@mail.gmail.com>

Hey Rory,

Yes, with b152 it works much better.

I did however found another somewhat related issue.
When compiling code that is using sun.misc.Unsafe (or any other class in
jdk.unsupported module)
and using -release 8 (and -source 8 -target 8) compile will fail with CNFE.
If I remove -release 8 it works fine.
It also works fine with -release 9 and --add-modules=jdk.unsupported but
this is different scenario.

Should I create issue for this? As problem is only when you use -release 8,
without it works just fine.

--
tomaz




On Wed, Jan 18, 2017 at 10:54 AM, Rory O'Donnell <rory.odonnell at oracle.com>
wrote:

> Hi Tomaz,
>
> The fix is in b152, available now,  let us know if that works ?
>
> Rgds,Rory
>
> On 10/01/2017 10:48, Toma? Cerar wrote:
>
> Hey Rory,
>
> Best wishes for new year also to you and your team.
>
> we already tested b150 and b151 but our testing is mostly blocked by
> already fixed https://bugs.openjdk.java.net/browse/JDK-8172158
> which I am guessing should be part of upcoming b152.
>
> Beyond that I am cautiously optimistic about our work around JDK9 and
> passing our testsuite.
>
> --
> tomaz
>
> On Tue, Jan 10, 2017 at 11:20 AM, Rory O'Donnell <rory.odonnell at oracle.com
> > wrote:
>
>>
>> Hi Jason/Tomaz,
>>
>> Best wishes for the New Year.
>>
>> Dalibor and I will be at FOSDEM '17, Brussels 4 & 5 February. Let us know
>> if you will be there, hopefully we can meet up !
>> *JDK 9 Early Access*  b151 <https://jdk9.java.net/download/>  is
>> available on java.net
>>
>> There have been a number of fixes to bugs reported by Open Source
>> projects since the last availability email  :
>>
>>    - JDK-8171377 : Add sun.misc.Unsafe::invokeCleaner
>>    - JDK-8075793 : Source incompatibility for inference using -source 7
>>    - JDK-8087303  : LSSerializer pretty print does not work anymore
>>    - JDK-8167143 :CLDR timezone parsing does not work for all locales
>>
>> Other changes that maybe of interest:
>>
>>    - JDK-8066474 : Remove the lib/$ARCH directory from Linux and Solaris
>>    images
>>    - JDK-8170428 : Move src.zip to JDK/lib/src.zip
>>
>> *JEPs intergrated:*
>>
>>    -  JEP 295 <http://openjdk.java.net/jeps/295>: Ahead-of-Time
>>    Compilation has been integrated in b150.
>>
>> *Schedule - Milestones since last availability email *
>>
>>    - *Feature Extension Complete 22nd of December 2016*
>>    -
>> *Rampdown Started 5th of January 2017 *
>>       - Phases in which increasing levels of scrutiny are applied to
>>       incoming changes.
>>       - In phase 1, only P1-P3 bugs can be fixed. In phase 2 only
>>       showstopper bugs can be fixed.
>>
>> Rgds,Rory
>>
>> --
>> Rgds,Rory O'Donnell
>> Quality Engineering Manager
>> Oracle EMEA , Dublin, Ireland
>>
>>
>
> --
> Rgds,Rory O'Donnell
> Quality Engineering Manager
> Oracle EMEA , Dublin, Ireland
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170118/32d5700b/attachment.html 

From rory.odonnell at oracle.com  Wed Jan 18 09:28:35 2017
From: rory.odonnell at oracle.com (Rory O'Donnell)
Date: Wed, 18 Jan 2017 14:28:35 +0000
Subject: [wildfly-dev] JDK 9 EA Build 151 is available on java.net
In-Reply-To: <CAMquZP6pdQMvdovJ1Ra0DCMHiJqiXEbpmhr9kAw7eXULw33PuA@mail.gmail.com>
References: <dcb219ec-6b44-1cf6-da23-1089c95a7233@oracle.com>
	<CAMquZP7Fui+ddzw3uLbN11Wg7vw-JT-Hv48dbs_OWDjrQw8W1A@mail.gmail.com>
	<e7ac4c6c-88d9-92d2-32fd-dff0371ef087@oracle.com>
	<CAMquZP6pdQMvdovJ1Ra0DCMHiJqiXEbpmhr9kAw7eXULw33PuA@mail.gmail.com>
Message-ID: <9f368991-216e-68ee-f272-af10aaf899df@oracle.com>



On 18/01/2017 13:25, Toma? Cerar wrote:
> Hey Rory,
>
> Yes, with b152 it works much better.
>
> I did however found another somewhat related issue.
> When compiling code that is using sun.misc.Unsafe (or any other class 
> in jdk.unsupported module)
> and using -release 8 (and -source 8 -target 8) compile will fail with 
> CNFE.
> If I remove -release 8 it works fine.
> It also works fine with -release 9 and --add-modules=jdk.unsupported 
> but this is different scenario.
>
> Should I create issue for this? As problem is only when you use 
> -release 8, without it works just fine.
Yes please, send us the JI

Thanks,Rory
>
> --
> tomaz
>
>
>
>
> On Wed, Jan 18, 2017 at 10:54 AM, Rory O'Donnell 
> <rory.odonnell at oracle.com <mailto:rory.odonnell at oracle.com>> wrote:
>
>     Hi Tomaz,
>
>     The fix is in b152, available now,  let us know if that works ?
>
>     Rgds,Rory
>
>
>     On 10/01/2017 10:48, Toma? Cerar wrote:
>>     Hey Rory,
>>
>>     Best wishes for new year also to you and your team.
>>
>>     we already tested b150 and b151 but our testing is mostly blocked
>>     by already fixed https://bugs.openjdk.java.net/browse/JDK-8172158
>>     <https://bugs.openjdk.java.net/browse/JDK-8172158>
>>     which I am guessing should be part of upcoming b152.
>>
>>     Beyond that I am cautiously optimistic about our work around JDK9
>>     and passing our testsuite.
>>
>>     --
>>     tomaz
>>
>>     On Tue, Jan 10, 2017 at 11:20 AM, Rory O'Donnell
>>     <rory.odonnell at oracle.com <mailto:rory.odonnell at oracle.com>> wrote:
>>
>>
>>         Hi Jason/Tomaz,
>>
>>         Best wishes for the New Year.
>>
>>         Dalibor and I will be at FOSDEM '17, Brussels 4 & 5 February.
>>         Let us know if you will be there, hopefully we can meet up !
>>
>>         *JDK 9 Early Access* b151 <https://jdk9.java.net/download/> 
>>         is available on java.net <http://java.net>
>>
>>         There have been a number of fixes to bugs reported by Open
>>         Source projects since the last availability email  :
>>
>>           * JDK-8171377 : Add sun.misc.Unsafe::invokeCleaner
>>           * JDK-8075793 : Source incompatibility for inference using
>>             -source 7
>>           * JDK-8087303  : LSSerializer pretty print does not work
>>             anymore
>>           * JDK-8167143 :CLDR timezone parsing does not work for all
>>             locales
>>
>>         Other changes that maybe of interest:
>>
>>           * JDK-8066474 : Remove the lib/$ARCH directory from Linux
>>             and Solaris images
>>           * JDK-8170428 : Move src.zip to JDK/lib/src.zip
>>
>>         *JEPs intergrated:*
>>
>>           * JEP 295 <http://openjdk.java.net/jeps/295>: Ahead-of-Time
>>             Compilation has been integrated in b150.
>>
>>         *Schedule - Milestones since last availability email *
>>
>>           * *Feature Extension Complete 22nd of December 2016*
>>           * *Rampdown Started 5th of January 2017
>>             *
>>               o Phases in which increasing levels of scrutiny are
>>                 applied to incoming changes.
>>               o In phase 1, only P1-P3 bugs can be fixed. In phase 2
>>                 only showstopper bugs can be fixed.
>>
>>         Rgds,Rory
>>
>>         -- 
>>         Rgds,Rory O'Donnell
>>         Quality Engineering Manager
>>         Oracle EMEA , Dublin, Ireland
>>
>>
>
>     -- 
>     Rgds,Rory O'Donnell
>     Quality Engineering Manager
>     Oracle EMEA , Dublin, Ireland
>
>

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170118/e6b4b5ec/attachment.html 

From jkasik at redhat.com  Mon Jan 23 05:51:23 2017
From: jkasik at redhat.com (Jan Kasik)
Date: Mon, 23 Jan 2017 11:51:23 +0100
Subject: [wildfly-dev] Always fixed thread pool in IO subsystem's worker
Message-ID: <CA+UTtG9Y63GWsaQXtjXNuT78K-SypfFb1MjcxxwWJ=fPzk00mg@mail.gmail.com>

Hi, I recently started writing test for newly exposed metrics from
underlying XNIO to IO subsystem (https://issues.jboss.org/browse/WFCORE-1341).
Because I didn't understand the values I dig into XNIO implementation where
I found out that the underlying thread pool in org.xnio.XnioWorker is
always fixed size (e.g.: 'corePoolSize == maxPoolSize' - see line 117). I
already talked with David M. Lloyd about this and from what he said I
understood that this is currently feature because of danger of race
conditions.
Because of this state there are several issues in context of IO subsystem
which bother me and I am not sure what to think about it:
* Newly exposed metric 'max-pool-size' is same value as already present
'task-max-threads' - are they really duplicitous?
* Newly exposed metrics 'max-pool-size' and 'core-pool-size' are always
equal
* There is no way for user to find out without studying implementation -
shouldn't be user informed about this?
* Since they are always fixed size, the attribute 'task-keepalive' has no
meaning - am I right?

Thank you for your ideas!

-- 
Jan (Honza) Kasik
Red Hat, Associate Quality Engineer, EAP QE
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20170123/dbef8149/attachment-0001.html 

From david.lloyd at redhat.com  Mon Jan 23 08:07:23 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Mon, 23 Jan 2017 07:07:23 -0600
Subject: [wildfly-dev] Always fixed thread pool in IO subsystem's worker
In-Reply-To: <CA+UTtG9Y63GWsaQXtjXNuT78K-SypfFb1MjcxxwWJ=fPzk00mg@mail.gmail.com>
References: <CA+UTtG9Y63GWsaQXtjXNuT78K-SypfFb1MjcxxwWJ=fPzk00mg@mail.gmail.com>
Message-ID: <1aa8856b-dad6-3153-fbb4-1ba712b7638c@redhat.com>

On 01/23/2017 04:51 AM, Jan Kasik wrote:
> Hi, I recently started writing test for newly exposed metrics from
> underlying XNIO to IO subsystem
> (https://issues.jboss.org/browse/WFCORE-1341). Because I didn't
> understand the values I dig into XNIO implementation where I found out
> that the underlying thread pool in org.xnio.XnioWorker is always fixed
> size (e.g.: 'corePoolSize == maxPoolSize' - see line 117). I already
> talked with David M. Lloyd about this and from what he said I understood
> that this is currently feature because of danger of race conditions.
> Because of this state there are several issues in context of IO
> subsystem which bother me and I am not sure what to think about it:
> * Newly exposed metric 'max-pool-size' is same value as already present
> 'task-max-threads' - are they really duplicitous?
> * Newly exposed metrics 'max-pool-size' and 'core-pool-size' are always
> equal
> * There is no way for user to find out without studying implementation -
> shouldn't be user informed about this?
> * Since they are always fixed size, the attribute 'task-keepalive' has
> no meaning - am I right?

For now.  It is on my task list to improve the thread pool 
implementation, at which point these synonymous attributes will start 
having individual meaning.

-- 
- DML

From david.lloyd at redhat.com  Mon Jan 23 11:59:36 2017
From: david.lloyd at redhat.com (David M. Lloyd)
Date: Mon, 23 Jan 2017 10:59:36 -0600
Subject: [wildfly-dev] The final (?) property expression expander
Message-ID: <68404560-a4b5-5820-87df-f5614e669ab2@redhat.com>

The basic problem is that we have a variety of client libraries that 
need property expansion, which are rolling their own or not doing it at 
all at present.  We have a couple implementations of property expansion 
on the server.  We have some potential future property expansion 
requirements.  Some or all of these things do (or need to do) property 
expansion slightly differently.  We've been balling up and depositing 
the same properties code over and over again, evolving it slightly each 
time, which in turn makes it harder to adapt to the next use case.  So, 
it's time to stop the madness.

In wildfly-common I'm introducing a new properties expander, implemented 
as a pure recursive-descent parser instead of the previous NFA-ish 
parser.  It is divided into two parts: syntax and expansion.

Syntax is handled by the expression compiler, whose API consists of a 
static factory method that accepts a pattern string and syntax flags, 
and returns the resultant Expression.

The syntax flags currently allow for the following syntax behaviors:

? NO_TRIM: Do not trim leading and trailing whitespace off of the 
expression string before parsing it.
? LENIENT_SYNTAX: Ignore syntax problems whenever possible instead of 
throwing an exception.
? MINI_EXPRS: Support single-character expressions that can be 
interpreted without wrapping in curly braces.
? NO_RECURSE_KEY: Do not support recursive expression expansion in the 
key part of the expression.
? NO_RECURSE_DEFAULT: Do not support recursion in default values.
? NO_SMART_BRACES: Do not support smart braces (this is where you have 
{something} inside of a key or default value).
? GENERAL_EXPANSION: Support Policy file style "general" expansion 
alternate expression syntax.  "Smart" braces will only work if the 
opening brace is not the first character in the expression key.
? ESCAPES: Support standard Java escape sequences in plain text and 
default value fields, which begin with a backslash character.
? DOUBLE_COLON: Treat expressions containing a double-colon initial 
delimiter as special, encoding the entire content into the key.

More behaviors can be contributed (along with corresponding tests of 
course).

Once an Expression is compiled, the resultant object can be used for 
expansion by providing an expansion function.  The function is given a 
context which allows introspection into the key sub-expression, the 
default value sub-expression, and the string builder target.  In 
addition, the function may throw at most one checked exception type of 
the user's choice, allowing expansion problems to be reported in any way.

The API provides a few default expanders to support only simple system 
properties and environment variables in the de-facto standard manner 
that we have always done; this is useful for client libraries with basic 
behavior.

Find the initial code here: 
https://github.com/wildfly/wildfly-common/pull/10

I'll merge it pending a bit more testing & any feedback.
-- 
- DML

From ppalaga at redhat.com  Wed Jan 25 06:39:56 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Wed, 25 Jan 2017 12:39:56 +0100
Subject: [wildfly-dev] Maven 3.3.1+ coming to wildfly and wildfly-core
In-Reply-To: <6ce4cc4c-2661-7901-4fd1-9f96dd14029a@redhat.com>
References: <6d206f5a-a45a-6bcb-1deb-2f4feee34ba2@redhat.com>
	<CADGwLpi_rGjn9rCSLMVy6YtD2Z1zs2jcjd8LpAoXE=FFY59nMg@mail.gmail.com>
	<f55703ff-7a5b-4bf9-79bf-54936efb253b@redhat.com>
	<CAJtdNN6wgL7xzGOEB+xmBpzj2x85KEQJ_M+ojxU6HNL3JqQGOQ@mail.gmail.com>
	<b10e0573-d1bc-e631-a48e-1318bf7dcb60@redhat.com>
	<6ce4cc4c-2661-7901-4fd1-9f96dd14029a@redhat.com>
Message-ID: <a512c25e-ea98-abd1-d164-e6f34d479e41@redhat.com>

Hi *,

I have just sent the PR to wildfly: 
https://github.com/wildfly/wildfly/pull/9577

As agreed here on wildfly-dev, the PR adapts the build.* and 
integration-tests.* scripts to use mvnw and keeps those old scripts for 
backwards compatibility.
The README was changed to recommend plain mvn or mvnw rather than the 
old build.* scripts.

Scott, James, Petr and others please review and eventually test in your 
scenarios.

Thanks,

Peter

On 2017-01-12 09:53, Peter Palaga wrote:
> The PR https://github.com/wildfly/wildfly-core/pull/2017 got merged in
> WF Core (thanks Brian!) and I am going to prepare a similar PR for WF. -- P
>
> On 2016-12-14 09:36, Peter Palaga wrote:
>> Hi Scott, OK, let's keep the build.(sh|bat) scripts for now. I'll change
>> them (1) to wrap mvnw and (2) to print a warning that they are
>> deprecated in favor of mvnw. -- P
>>
>> On 2016-12-14 01:04, Scott Marlow wrote:
>>> Hi Peter,
>>>
>>> On Tue, Dec 13, 2016 at 5:26 PM, Peter Palaga <ppalaga at redhat.com> wrote:
>>>> Hi James,
>>>>
>>>> On 2016-12-13 22:51, James Perkins wrote:
>>>>> Hi Peter,
>>>>> It may not be a big deal because it's just for builds, but instead of
>>>>> deleting the build.(sh|bat) maybe we just make those invoke the
>>>>> appropriate mvnw command instead.
>>>>
>>>> That would be easy to do, but can you name a situation where it would
>>>> actually matter?
>>>
>>> We have a few (compliance) test scripts that invoke build.sh to build
>>> the app server from source.  I'm not sure how widespread use of
>>> build.sh is in other test environments but we probably could ask that
>>> question via a separate topic.
>>>
>>> Scott
>>>
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


From ppalaga at redhat.com  Wed Jan 25 16:45:44 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Wed, 25 Jan 2017 22:45:44 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
Message-ID: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>

Hi *,

this is not new to those of you who attended my talk on the F2F 2016 in 
Brno. Let me explain the idea here again for all others who did not have 
a chance to be there.

Srcdeps [1] is a tool to build Maven dependencies from their sources. 
With srcdeps, wildfly-core can depend on a specific commit of, e.g., 
undertow:

<version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>

where aabbccd is the git commit id to build when any undertow artifact 
is requested during the build of wildfly-core.

[1] describes in detail, how it works.

The main advantage of srcdeps is that changes in components can be 
integrated and tested in wildfly-core immediately after they are 
committed to a public component branch. There is no need to wait for the 
component release.

Here in the WildFly family of projects, it is often the case that 
something needs to be fixed in a component, but the verification (using 
bug reproducer, or integration test) is possible only at the level of 
wildfly or wildfly-core. Engineers typically work with snapshots 
locally, but when their changes need to get shared (CI, reviews) in a 
reproducible manner, snapshots cannot be used anymore.
In such situations a source dependency come in handy: it is very easy to 
share and it is as reproducible as a Maven build from a specific commit 
can be. All CIs and reviewers can work with it, because all source 
dependency compilation is done under the hood by Maven.

Developers working on changes that span over multiple interdependent git 
repos can thus get feedback (i-tests, reviews) quickly without waiting 
for releases of components.

Srcdeps emerged in the Hawkular family of projects to solve exactly this 
kind of situation and is in use there since around October 2015.

When I said there is no need to wait for releases of components, I did 
not mean that we can get rid of component releases altogether. Clearly, 
we cannot, because i.a. for any tooling uninformed about how srcdeps 
work, those source dependencies would simply be non-resolvable from 
public Maven repositories. So, before releasing the dependent component 
(such as wildfly-core) all its dependencies need to be released. To 
enforce this, srcdeps is by default configured to make the release fail, 
as long as there are source dependencies.

I have sent a PR introducing srcdeps to wildfly-core: 
https://github.com/wildfly/wildfly-core/pull/2122
To get a feeling how it works, checkout the branch, switch to e.g. 
<version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow> 
(that happens to be the commit id of the 1.4.8.Final tag)
and build wildfly-core as usual with "mvn clean install". You'll see in 
the build log that undertow is being cloned to ~/.m2/srcdeps/io/undertow 
and that it is built there. After the build, check that the 
1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd 
version of Undertow got installed to your local Maven repo (usually 
~/m2/repository/io/undertow/undertow-core )

Are there any questions or comments?

[1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven

Thanks,

Peter

P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at 14:30 at 
DevConf Brno.

From brian.stansberry at redhat.com  Wed Jan 25 20:02:16 2017
From: brian.stansberry at redhat.com (Brian Stansberry)
Date: Wed, 25 Jan 2017 19:02:16 -0600
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
Message-ID: <BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>

My only concerns with this would relate to comitting this kind of src dependency to the poms in the main branches in the widlfly/wildfly and wildfly/wildfly-core repos. We?ve managed to survive up to now with little or no need for that kind of thing, so until we get used to using this in other ways IMHO we should follow the KISS principle and forbid that.

A trick is avoiding doing that by mistake; i.e. a PR is sent up with a SRC dependency to get CI or review and accidentally gets merged. But I suppose that?s not the end of the world, so long as the release process will eventually detect it and fail.

Can making srcdeps fail (or just disabling it) be turned on via a maven profile? With that we could set up such a profile and turn it on in CI jobs that are testing branches where it?s forbidden (e.g. the nightly builds of master.)

Oh, one other concern ? how robust is this in the face of poor maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml. If that gets out of date or something is the only effect that using a src dependency for the affected item doesn't work?

> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com> wrote:
> 
> Hi *,
> 
> this is not new to those of you who attended my talk on the F2F 2016 in 
> Brno. Let me explain the idea here again for all others who did not have 
> a chance to be there.
> 
> Srcdeps [1] is a tool to build Maven dependencies from their sources. 
> With srcdeps, wildfly-core can depend on a specific commit of, e.g., 
> undertow:
> 
> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
> 
> where aabbccd is the git commit id to build when any undertow artifact 
> is requested during the build of wildfly-core.
> 
> [1] describes in detail, how it works.
> 
> The main advantage of srcdeps is that changes in components can be 
> integrated and tested in wildfly-core immediately after they are 
> committed to a public component branch. There is no need to wait for the 
> component release.
> 
> Here in the WildFly family of projects, it is often the case that 
> something needs to be fixed in a component, but the verification (using 
> bug reproducer, or integration test) is possible only at the level of 
> wildfly or wildfly-core. Engineers typically work with snapshots 
> locally, but when their changes need to get shared (CI, reviews) in a 
> reproducible manner, snapshots cannot be used anymore.
> In such situations a source dependency come in handy: it is very easy to 
> share and it is as reproducible as a Maven build from a specific commit 
> can be. All CIs and reviewers can work with it, because all source 
> dependency compilation is done under the hood by Maven.
> 
> Developers working on changes that span over multiple interdependent git 
> repos can thus get feedback (i-tests, reviews) quickly without waiting 
> for releases of components.
> 
> Srcdeps emerged in the Hawkular family of projects to solve exactly this 
> kind of situation and is in use there since around October 2015.
> 
> When I said there is no need to wait for releases of components, I did 
> not mean that we can get rid of component releases altogether. Clearly, 
> we cannot, because i.a. for any tooling uninformed about how srcdeps 
> work, those source dependencies would simply be non-resolvable from 
> public Maven repositories. So, before releasing the dependent component 
> (such as wildfly-core) all its dependencies need to be released. To 
> enforce this, srcdeps is by default configured to make the release fail, 
> as long as there are source dependencies.
> 
> I have sent a PR introducing srcdeps to wildfly-core: 
> https://github.com/wildfly/wildfly-core/pull/2122
> To get a feeling how it works, checkout the branch, switch to e.g. 
> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow> 
> (that happens to be the commit id of the 1.4.8.Final tag)
> and build wildfly-core as usual with "mvn clean install". You'll see in 
> the build log that undertow is being cloned to ~/.m2/srcdeps/io/undertow 
> and that it is built there. After the build, check that the 
> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd 
> version of Undertow got installed to your local Maven repo (usually 
> ~/m2/repository/io/undertow/undertow-core )
> 
> Are there any questions or comments?
> 
> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
> 
> Thanks,
> 
> Peter
> 
> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at 14:30 at 
> DevConf Brno.
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat





From ppalaga at redhat.com  Thu Jan 26 03:54:45 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Thu, 26 Jan 2017 09:54:45 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
Message-ID: <d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>

Hi Brian, thanks for your comments, more inline...

On 2017-01-26 02:02, Brian Stansberry wrote:
> My only concerns with this would relate to comitting this kind of src
> dependency to the poms in the main branches in the widlfly/wildfly
> and wildfly/wildfly-core repos. We?ve managed to survive up to now
> with little or no need for that kind of thing, so until we get used
> to using this in other ways IMHO we should follow the KISS principle
> and forbid that.

Maybe I overestimate the amount of changes that span over multiple git 
repos. Maybe you in the Core team do not do this often. But for us in 
the Sustaining Engineering Team, this is quite a typical situation. A 
substantial part of the reports from customers come with a description 
how to reproduce on the whole server, but they need to be fixed in a 
component. Having srcdeps would make the CP process simpler and faster, 
allowing us to uncover the conflicts and regressions earlier.

> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
> a SRC dependency to get CI or review and accidentally gets merged.

Oh, I am just realizing I have not said anything about merging. I 
actually do want to propose that commits with source dependencies get 
merged to e.g. wildfly-core master as early as possible. Those are the 
key points of Continuous Integration: get feedback quickly, and merge as 
soon as possible. This is exactly what Hawkular is doing since more than 
a year.

> But I suppose that?s not the end of the world, so long as the release
> process will eventually detect it and fail.

Yes, source dependencies on a stable branch do not harm. They just need 
to be avoided in releases (for which srcdeps offers technical means).

> Can making srcdeps fail (or just disabling it) be turned on via a
> maven profile? With that we could set up such a profile and turn it
> on in CI jobs that are testing branches where it?s forbidden (e.g.
> the nightly builds of master.)

Yes, the feature is called "failWith profiles" and can be configured in 
.mvn/srcdeps.yaml, like here in this srcdeps quickstart: 
https://github.com/srcdeps/srcdeps-maven/blob/master/srcdeps-maven-quickstarts/srcdeps-mvn-git-profile-quickstart/.mvn/srcdeps.yaml#L33 

There is also "failWith properties" and "failWith goals". It is 
documented here: 
https://github.com/srcdeps/srcdeps-core/blob/master/doc/srcdeps.yaml#L130
By default there is failWith: {goals: release:prepare, release:perform}. 
Projects that do not use the release plugin can set e.g. failWith: 
{goals: deploy:deploy} or whatever else distinguishes their releases.

> Oh, one other concern ? how robust is this in the face of poor
> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml.

Which parts are boilerpate?

> If
> that gets out of date or something is the only effect that using a
> src dependency for the affected item doesn't work?

Yes, I think so. As long as the .mvn/srcdeps.yaml file is syntactically 
correct, any misconfiguration there should not have any other effect 
than eventually breaking an embedded build.

Generally, the things configured in .mvn/srcdeps.yaml tend to be quite 
stable - it is basically just mapping from GAVs to their respective git 
URLs. Git URLs do not change often. It is true that dependency artifacts 
come and go, but as long as their groupIds are selected reasonably (one 
groupId occurs in not more than one git repo) the mapping itself can be 
quite stable over time too.

Thanks,

Peter

>
>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com>
>> wrote:
>>
>> Hi *,
>>
>> this is not new to those of you who attended my talk on the F2F
>> 2016 in Brno. Let me explain the idea here again for all others who
>> did not have a chance to be there.
>>
>> Srcdeps [1] is a tool to build Maven dependencies from their
>> sources. With srcdeps, wildfly-core can depend on a specific commit
>> of, e.g., undertow:
>>
>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>
>>
>>
where aabbccd is the git commit id to build when any undertow artifact
>> is requested during the build of wildfly-core.
>>
>> [1] describes in detail, how it works.
>>
>> The main advantage of srcdeps is that changes in components can be
>>  integrated and tested in wildfly-core immediately after they are
>> committed to a public component branch. There is no need to wait
>> for the component release.
>>
>> Here in the WildFly family of projects, it is often the case that
>> something needs to be fixed in a component, but the verification
>> (using bug reproducer, or integration test) is possible only at the
>> level of wildfly or wildfly-core. Engineers typically work with
>> snapshots locally, but when their changes need to get shared (CI,
>> reviews) in a reproducible manner, snapshots cannot be used
>> anymore. In such situations a source dependency come in handy: it
>> is very easy to share and it is as reproducible as a Maven build
>> from a specific commit can be. All CIs and reviewers can work with
>> it, because all source dependency compilation is done under the
>> hood by Maven.
>>
>> Developers working on changes that span over multiple
>> interdependent git repos can thus get feedback (i-tests, reviews)
>> quickly without waiting for releases of components.
>>
>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>> this kind of situation and is in use there since around October
>> 2015.
>>
>> When I said there is no need to wait for releases of components, I
>> did not mean that we can get rid of component releases altogether.
>> Clearly, we cannot, because i.a. for any tooling uninformed about
>> how srcdeps work, those source dependencies would simply be
>> non-resolvable from public Maven repositories. So, before releasing
>> the dependent component (such as wildfly-core) all its dependencies
>> need to be released. To enforce this, srcdeps is by default
>> configured to make the release fail, as long as there are source
>> dependencies.
>>
>> I have sent a PR introducing srcdeps to wildfly-core:
>> https://github.com/wildfly/wildfly-core/pull/2122 To get a feeling
>> how it works, checkout the branch, switch to e.g.
>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>  (that happens to be the commit id of the 1.4.8.Final tag) and
>> build wildfly-core as usual with "mvn clean install". You'll see in
>>  the build log that undertow is being cloned to
>> ~/.m2/srcdeps/io/undertow and that it is built there. After the
>> build, check that the
>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>> version of Undertow got installed to your local Maven repo (usually
>>  ~/m2/repository/io/undertow/undertow-core )
>>
>> Are there any questions or comments?
>>
>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>
>> Thanks,
>>
>> Peter
>>
>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>> 14:30 at DevConf Brno.
>> _______________________________________________ wildfly-dev mailing
>> list wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


From psakar at redhat.com  Thu Jan 26 04:52:54 2017
From: psakar at redhat.com (Petr Sakar)
Date: Thu, 26 Jan 2017 10:52:54 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
Message-ID: <fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>



On 01/26/2017 09:54 AM, Peter Palaga wrote:
> Hi Brian, thanks for your comments, more inline...
>
> On 2017-01-26 02:02, Brian Stansberry wrote:
>> My only concerns with this would relate to comitting this kind of src
>> dependency to the poms in the main branches in the widlfly/wildfly
>> and wildfly/wildfly-core repos. We?ve managed to survive up to now
>> with little or no need for that kind of thing, so until we get used
>> to using this in other ways IMHO we should follow the KISS principle
>> and forbid that.
> Maybe I overestimate the amount of changes that span over multiple git
> repos. Maybe you in the Core team do not do this often. But for us in
> the Sustaining Engineering Team, this is quite a typical situation. A
> substantial part of the reports from customers come with a description
> how to reproduce on the whole server, but they need to be fixed in a
> component. Having srcdeps would make the CP process simpler and faster,
> allowing us to uncover the conflicts and regressions earlier.
>
>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>> a SRC dependency to get CI or review and accidentally gets merged.
> Oh, I am just realizing I have not said anything about merging. I
> actually do want to propose that commits with source dependencies get
> merged to e.g. wildfly-core master as early as possible. Those are the
> key points of Continuous Integration: get feedback quickly, and merge as
> soon as possible. This is exactly what Hawkular is doing since more than
> a year.

I agree with Brian. PR with SRC dependency *should not* be merged to master branch, that would create problems down the road.
You can use other branch as base for such merges and related CI
Petr

>> But I suppose that?s not the end of the world, so long as the release
>> process will eventually detect it and fail.
> Yes, source dependencies on a stable branch do not harm. They just need
> to be avoided in releases (for which srcdeps offers technical means).
>
>> Can making srcdeps fail (or just disabling it) be turned on via a
>> maven profile? With that we could set up such a profile and turn it
>> on in CI jobs that are testing branches where it?s forbidden (e.g.
>> the nightly builds of master.)
> Yes, the feature is called "failWith profiles" and can be configured in
> .mvn/srcdeps.yaml, like here in this srcdeps quickstart:
> https://github.com/srcdeps/srcdeps-maven/blob/master/srcdeps-maven-quickstarts/srcdeps-mvn-git-profile-quickstart/.mvn/srcdeps.yaml#L33
>
> There is also "failWith properties" and "failWith goals". It is
> documented here:
> https://github.com/srcdeps/srcdeps-core/blob/master/doc/srcdeps.yaml#L130
> By default there is failWith: {goals: release:prepare, release:perform}.
> Projects that do not use the release plugin can set e.g. failWith:
> {goals: deploy:deploy} or whatever else distinguishes their releases.
>
>> Oh, one other concern ? how robust is this in the face of poor
>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml.
> Which parts are boilerpate?
>
>> If
>> that gets out of date or something is the only effect that using a
>> src dependency for the affected item doesn't work?
> Yes, I think so. As long as the .mvn/srcdeps.yaml file is syntactically
> correct, any misconfiguration there should not have any other effect
> than eventually breaking an embedded build.
>
> Generally, the things configured in .mvn/srcdeps.yaml tend to be quite
> stable - it is basically just mapping from GAVs to their respective git
> URLs. Git URLs do not change often. It is true that dependency artifacts
> come and go, but as long as their groupIds are selected reasonably (one
> groupId occurs in not more than one git repo) the mapping itself can be
> quite stable over time too.
>
> Thanks,
>
> Peter
>
>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com>
>>> wrote:
>>>
>>> Hi *,
>>>
>>> this is not new to those of you who attended my talk on the F2F
>>> 2016 in Brno. Let me explain the idea here again for all others who
>>> did not have a chance to be there.
>>>
>>> Srcdeps [1] is a tool to build Maven dependencies from their
>>> sources. With srcdeps, wildfly-core can depend on a specific commit
>>> of, e.g., undertow:
>>>
>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>
>>>
>>>
> where aabbccd is the git commit id to build when any undertow artifact
>>> is requested during the build of wildfly-core.
>>>
>>> [1] describes in detail, how it works.
>>>
>>> The main advantage of srcdeps is that changes in components can be
>>>   integrated and tested in wildfly-core immediately after they are
>>> committed to a public component branch. There is no need to wait
>>> for the component release.
>>>
>>> Here in the WildFly family of projects, it is often the case that
>>> something needs to be fixed in a component, but the verification
>>> (using bug reproducer, or integration test) is possible only at the
>>> level of wildfly or wildfly-core. Engineers typically work with
>>> snapshots locally, but when their changes need to get shared (CI,
>>> reviews) in a reproducible manner, snapshots cannot be used
>>> anymore. In such situations a source dependency come in handy: it
>>> is very easy to share and it is as reproducible as a Maven build
>>> from a specific commit can be. All CIs and reviewers can work with
>>> it, because all source dependency compilation is done under the
>>> hood by Maven.
>>>
>>> Developers working on changes that span over multiple
>>> interdependent git repos can thus get feedback (i-tests, reviews)
>>> quickly without waiting for releases of components.
>>>
>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>> this kind of situation and is in use there since around October
>>> 2015.
>>>
>>> When I said there is no need to wait for releases of components, I
>>> did not mean that we can get rid of component releases altogether.
>>> Clearly, we cannot, because i.a. for any tooling uninformed about
>>> how srcdeps work, those source dependencies would simply be
>>> non-resolvable from public Maven repositories. So, before releasing
>>> the dependent component (such as wildfly-core) all its dependencies
>>> need to be released. To enforce this, srcdeps is by default
>>> configured to make the release fail, as long as there are source
>>> dependencies.
>>>
>>> I have sent a PR introducing srcdeps to wildfly-core:
>>> https://github.com/wildfly/wildfly-core/pull/2122 To get a feeling
>>> how it works, checkout the branch, switch to e.g.
>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>>   (that happens to be the commit id of the 1.4.8.Final tag) and
>>> build wildfly-core as usual with "mvn clean install". You'll see in
>>>   the build log that undertow is being cloned to
>>> ~/.m2/srcdeps/io/undertow and that it is built there. After the
>>> build, check that the
>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>> version of Undertow got installed to your local Maven repo (usually
>>>   ~/m2/repository/io/undertow/undertow-core )
>>>
>>> Are there any questions or comments?
>>>
>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>
>>> Thanks,
>>>
>>> Peter
>>>
>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>> 14:30 at DevConf Brno.
>>> _______________________________________________ wildfly-dev mailing
>>> list wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


From ppalaga at redhat.com  Thu Jan 26 04:56:56 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Thu, 26 Jan 2017 10:56:56 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
Message-ID: <0ff13127-2e92-9470-dce7-60ecb953dee5@redhat.com>

Ahoj Pet?e, which particular problems would be caused by a SRC 
dependency merged to a master branch? -- P

On 2017-01-26 10:52, Petr Sakar wrote:
>
>
> On 01/26/2017 09:54 AM, Peter Palaga wrote:
>> Hi Brian, thanks for your comments, more inline...
>>
>> On 2017-01-26 02:02, Brian Stansberry wrote:
>>> My only concerns with this would relate to comitting this kind of src
>>> dependency to the poms in the main branches in the widlfly/wildfly
>>> and wildfly/wildfly-core repos. We?ve managed to survive up to now
>>> with little or no need for that kind of thing, so until we get used
>>> to using this in other ways IMHO we should follow the KISS principle
>>> and forbid that.
>> Maybe I overestimate the amount of changes that span over multiple git
>> repos. Maybe you in the Core team do not do this often. But for us in
>> the Sustaining Engineering Team, this is quite a typical situation. A
>> substantial part of the reports from customers come with a description
>> how to reproduce on the whole server, but they need to be fixed in a
>> component. Having srcdeps would make the CP process simpler and faster,
>> allowing us to uncover the conflicts and regressions earlier.
>>
>>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>>> a SRC dependency to get CI or review and accidentally gets merged.
>> Oh, I am just realizing I have not said anything about merging. I
>> actually do want to propose that commits with source dependencies get
>> merged to e.g. wildfly-core master as early as possible. Those are the
>> key points of Continuous Integration: get feedback quickly, and merge as
>> soon as possible. This is exactly what Hawkular is doing since more than
>> a year.
>
> I agree with Brian. PR with SRC dependency *should not* be merged to master branch, that would create problems down the road.
> You can use other branch as base for such merges and related CI
> Petr
>
>>> But I suppose that?s not the end of the world, so long as the release
>>> process will eventually detect it and fail.
>> Yes, source dependencies on a stable branch do not harm. They just need
>> to be avoided in releases (for which srcdeps offers technical means).
>>
>>> Can making srcdeps fail (or just disabling it) be turned on via a
>>> maven profile? With that we could set up such a profile and turn it
>>> on in CI jobs that are testing branches where it?s forbidden (e.g.
>>> the nightly builds of master.)
>> Yes, the feature is called "failWith profiles" and can be configured in
>> .mvn/srcdeps.yaml, like here in this srcdeps quickstart:
>> https://github.com/srcdeps/srcdeps-maven/blob/master/srcdeps-maven-quickstarts/srcdeps-mvn-git-profile-quickstart/.mvn/srcdeps.yaml#L33
>>
>> There is also "failWith properties" and "failWith goals". It is
>> documented here:
>> https://github.com/srcdeps/srcdeps-core/blob/master/doc/srcdeps.yaml#L130
>> By default there is failWith: {goals: release:prepare, release:perform}.
>> Projects that do not use the release plugin can set e.g. failWith:
>> {goals: deploy:deploy} or whatever else distinguishes their releases.
>>
>>> Oh, one other concern ? how robust is this in the face of poor
>>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml.
>> Which parts are boilerpate?
>>
>>> If
>>> that gets out of date or something is the only effect that using a
>>> src dependency for the affected item doesn't work?
>> Yes, I think so. As long as the .mvn/srcdeps.yaml file is syntactically
>> correct, any misconfiguration there should not have any other effect
>> than eventually breaking an embedded build.
>>
>> Generally, the things configured in .mvn/srcdeps.yaml tend to be quite
>> stable - it is basically just mapping from GAVs to their respective git
>> URLs. Git URLs do not change often. It is true that dependency artifacts
>> come and go, but as long as their groupIds are selected reasonably (one
>> groupId occurs in not more than one git repo) the mapping itself can be
>> quite stable over time too.
>>
>> Thanks,
>>
>> Peter
>>
>>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com>
>>>> wrote:
>>>>
>>>> Hi *,
>>>>
>>>> this is not new to those of you who attended my talk on the F2F
>>>> 2016 in Brno. Let me explain the idea here again for all others who
>>>> did not have a chance to be there.
>>>>
>>>> Srcdeps [1] is a tool to build Maven dependencies from their
>>>> sources. With srcdeps, wildfly-core can depend on a specific commit
>>>> of, e.g., undertow:
>>>>
>>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>>
>>>>
>>>>
>> where aabbccd is the git commit id to build when any undertow artifact
>>>> is requested during the build of wildfly-core.
>>>>
>>>> [1] describes in detail, how it works.
>>>>
>>>> The main advantage of srcdeps is that changes in components can be
>>>>   integrated and tested in wildfly-core immediately after they are
>>>> committed to a public component branch. There is no need to wait
>>>> for the component release.
>>>>
>>>> Here in the WildFly family of projects, it is often the case that
>>>> something needs to be fixed in a component, but the verification
>>>> (using bug reproducer, or integration test) is possible only at the
>>>> level of wildfly or wildfly-core. Engineers typically work with
>>>> snapshots locally, but when their changes need to get shared (CI,
>>>> reviews) in a reproducible manner, snapshots cannot be used
>>>> anymore. In such situations a source dependency come in handy: it
>>>> is very easy to share and it is as reproducible as a Maven build
>>>> from a specific commit can be. All CIs and reviewers can work with
>>>> it, because all source dependency compilation is done under the
>>>> hood by Maven.
>>>>
>>>> Developers working on changes that span over multiple
>>>> interdependent git repos can thus get feedback (i-tests, reviews)
>>>> quickly without waiting for releases of components.
>>>>
>>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>>> this kind of situation and is in use there since around October
>>>> 2015.
>>>>
>>>> When I said there is no need to wait for releases of components, I
>>>> did not mean that we can get rid of component releases altogether.
>>>> Clearly, we cannot, because i.a. for any tooling uninformed about
>>>> how srcdeps work, those source dependencies would simply be
>>>> non-resolvable from public Maven repositories. So, before releasing
>>>> the dependent component (such as wildfly-core) all its dependencies
>>>> need to be released. To enforce this, srcdeps is by default
>>>> configured to make the release fail, as long as there are source
>>>> dependencies.
>>>>
>>>> I have sent a PR introducing srcdeps to wildfly-core:
>>>> https://github.com/wildfly/wildfly-core/pull/2122 To get a feeling
>>>> how it works, checkout the branch, switch to e.g.
>>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>>>   (that happens to be the commit id of the 1.4.8.Final tag) and
>>>> build wildfly-core as usual with "mvn clean install". You'll see in
>>>>   the build log that undertow is being cloned to
>>>> ~/.m2/srcdeps/io/undertow and that it is built there. After the
>>>> build, check that the
>>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>>> version of Undertow got installed to your local Maven repo (usually
>>>>   ~/m2/repository/io/undertow/undertow-core )
>>>>
>>>> Are there any questions or comments?
>>>>
>>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>>
>>>> Thanks,
>>>>
>>>> Peter
>>>>
>>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>>> 14:30 at DevConf Brno.
>>>> _______________________________________________ wildfly-dev mailing
>>>> list wildfly-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


From jcosta at redhat.com  Thu Jan 26 05:09:12 2017
From: jcosta at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=)
Date: Thu, 26 Jan 2017 11:09:12 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
Message-ID: <265cc9d0-c8db-354b-b380-4861caf7c5d5@redhat.com>

On 01/26/2017 10:52 AM, Petr Sakar wrote:
> I agree with Brian. PR with SRC dependency *should not* be merged to master branch, that would create problems down the road.
> You can use other branch as base for such merges and related CI

Let me share my experience with using srcdeps in Hawkular.

We've been bitten by this situation, where a srcdeps would be listed as 
a dependency and the release process would eventually fail (and it was 
an ugly failure). This is why there's a feature (on by default, IIRC), 
which fails fast if the release profile is being used.

A good practice in Hawkular Services is to _avoid_ having srcdeps, but 
it's _acceptable_ to have it on master, as long as it's removed by the 
release date. Most of the time, there's no srcdeps in our pom.xml files, 
but when we need it, it allows us to keep moving without asking N 
developers to release the whole dependency chain.

- Juca.

From kabir.khan at jboss.com  Thu Jan 26 05:25:13 2017
From: kabir.khan at jboss.com (Kabir Khan)
Date: Thu, 26 Jan 2017 10:25:13 +0000
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <265cc9d0-c8db-354b-b380-4861caf7c5d5@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
	<265cc9d0-c8db-354b-b380-4861caf7c5d5@redhat.com>
Message-ID: <E04A47DF-61A9-4EBD-AE5D-9AADD3B3FE5F@jboss.com>

Perhaps rather than a profile in the consuming project as Brian mentions, it should be disabled in srcdeps itself by default? Something along the lines that when its dependency resolution mechanism is hit, it fails unless -Dsrcdeps.enabled is passed in. Of course there is still a risk that someone adds that property to the pom by accident.

Also how does this work for nested projects? Say we add this to WildFly full, and I want to test WildFly with to something which is brought in by wildfly-core (e.g. undertow, Elytron etc.). How would that work?


> On 26 Jan 2017, at 10:09, Juraci Paix?o Kr?hling <jcosta at redhat.com> wrote:
> 
> On 01/26/2017 10:52 AM, Petr Sakar wrote:
>> I agree with Brian. PR with SRC dependency *should not* be merged to master branch, that would create problems down the road.
>> You can use other branch as base for such merges and related CI
> 
> Let me share my experience with using srcdeps in Hawkular.
> 
> We've been bitten by this situation, where a srcdeps would be listed as 
> a dependency and the release process would eventually fail (and it was 
> an ugly failure). This is why there's a feature (on by default, IIRC), 
> which fails fast if the release profile is being used.
> 
> A good practice in Hawkular Services is to _avoid_ having srcdeps, but 
> it's _acceptable_ to have it on master, as long as it's removed by the 
> release date. Most of the time, there's no srcdeps in our pom.xml files, 
> but when we need it, it allows us to keep moving without asking N 
> developers to release the whole dependency chain.
> 
> - Juca.
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev



From jcosta at redhat.com  Thu Jan 26 05:46:01 2017
From: jcosta at redhat.com (=?UTF-8?Q?Juraci_Paix=c3=a3o_Kr=c3=b6hling?=)
Date: Thu, 26 Jan 2017 11:46:01 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <E04A47DF-61A9-4EBD-AE5D-9AADD3B3FE5F@jboss.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
	<265cc9d0-c8db-354b-b380-4861caf7c5d5@redhat.com>
	<E04A47DF-61A9-4EBD-AE5D-9AADD3B3FE5F@jboss.com>
Message-ID: <08270852-7893-1241-f9e6-0cae171bcf0b@redhat.com>

On 01/26/2017 11:25 AM, Kabir Khan wrote:
> Perhaps rather than a profile in the consuming project as Brian mentions, it should be disabled in srcdeps itself by default? Something along the lines that when its dependency resolution mechanism is hit, it fails unless -Dsrcdeps.enabled is passed in. Of course there is still a risk that someone adds that property to the pom by accident.
>
> Also how does this work for nested projects? Say we add this to WildFly full, and I want to test WildFly with to something which is brought in by wildfly-core (e.g. undertow, Elytron etc.). How would that work?

It works quite nicely. I remember seeing 3 levels of nesting and I'm 
sure it could have even more.

It does delay the main build, though, as srcdeps builds the dependencies 
on demand. It means that the first local build of "wildfly-core" that 
depends on "elytron:1.2.3-src-abc123" will also build Elytron's revision 
"abc123".

- Juca.

From ppalaga at redhat.com  Thu Jan 26 06:37:09 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Thu, 26 Jan 2017 12:37:09 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <08270852-7893-1241-f9e6-0cae171bcf0b@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
	<265cc9d0-c8db-354b-b380-4861caf7c5d5@redhat.com>
	<E04A47DF-61A9-4EBD-AE5D-9AADD3B3FE5F@jboss.com>
	<08270852-7893-1241-f9e6-0cae171bcf0b@redhat.com>
Message-ID: <0823070b-3179-988d-5b55-385a711f479a@redhat.com>

On 2017-01-26 11:46, Juraci Paix?o Kr?hling wrote:
> On 01/26/2017 11:25 AM, Kabir Khan wrote:
>> Perhaps rather than a profile in the consuming project as Brian mentions, it should be disabled in srcdeps itself by default? Something along the lines that when its dependency resolution mechanism is hit, it fails unless -Dsrcdeps.enabled is passed in. Of course there is still a risk that someone adds that property to the pom by accident.

Not sure what would be the purpose of making srcdeps opt-in? - To allow 
CI and reviews, but to prevent merging? Well, could you tell me first 
what is wrong with merging source dependencies?

>> Also how does this work for nested projects? Say we add this to WildFly full, and I want to test WildFly with to something which is brought in by wildfly-core (e.g. undertow, Elytron etc.). How would that work?
>
> It works quite nicely. I remember seeing 3 levels of nesting and I'm
> sure it could have even more.
>
> It does delay the main build, though, as srcdeps builds the dependencies
> on demand. It means that the first local build of "wildfly-core" that
> depends on "elytron:1.2.3-src-abc123" will also build Elytron's revision
> "abc123".

Yes, as Juca said, running a build of project A that has a source 
dependency B, the build of B can trigger yet another build of its own 
source dependency C.

Perhaps I should note that the dependency builds run with -DskipTests by 
default so that they finish faster. For dependencies known to use 
checkstyle, license plugin or similar, their builds can be configured to 
skip those too to run even faster, as I do e.g. here 
https://github.com/wildfly/wildfly-core/pull/2122/commits/65326ef4ff6abc5673a605f2394003f9a9537fdb#diff-34efcdaa51afff46bba8cd9c1387e6f5R146

Further, once a source dependency is built, it is installed to Maven 
local repository and re-used from there for all subsequent builds that 
require it. A source dependency thus makes just the first build slower.
The local git repositories are kept too and therefore only the first 
clone takes long. All subsequent dependency builds use just fetch and reset.

-- P


From darran.lofthouse at jboss.com  Thu Jan 26 06:40:58 2017
From: darran.lofthouse at jboss.com (Darran Lofthouse)
Date: Thu, 26 Jan 2017 11:40:58 +0000
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
Message-ID: <cb3c9ece-98db-e1ba-af4f-738968d419af@jboss.com>

On 26/01/17 01:02, Brian Stansberry wrote:
> My only concerns with this would relate to comitting this kind of src dependency to the poms in the main branches in the widlfly/wildfly and wildfly/wildfly-core repos. We?ve managed to survive up to now with little or no need for that kind of thing, so until we get used to using this in other ways IMHO we should follow the KISS principle and forbid that.

I like the idea of this being available for teams working on a topic 
branch but also think we should avoid this kind of dependency in master 
- engineers are continually running builds so I don't think we should 
add additional source checkouts and builds.

I know Elytron has caused huge amounts of instability but generally I 
think we want these branches in a state where we could tag if we needed to.

When these downloaded builds are executed are the test cases within also 
executed?

> A trick is avoiding doing that by mistake; i.e. a PR is sent up with a SRC dependency to get CI or review and accidentally gets merged. But I suppose that?s not the end of the world, so long as the release process will eventually detect it and fail.

Merging topic branches these temporary SHAs would be in the history but 
maybe the PR CI runs could verify they are not present in the final merge.

> Can making srcdeps fail (or just disabling it) be turned on via a maven profile? With that we could set up such a profile and turn it on in CI jobs that are testing branches where it?s forbidden (e.g. the nightly builds of master.)
>
> Oh, one other concern ? how robust is this in the face of poor maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml. If that gets out of date or something is the only effect that using a src dependency for the affected item doesn't work?

I think that file would be better almost empty but not hidden.

Thinking about the topic branch development, within the Elytron team we 
are using some SNAPSHOTs from the official repos and at times we have 
forked projects into our incubator and developed against SNAPSHOTs from 
those as as you select the SHA for the build you would also need to 
select which repository you really want.

>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com> wrote:
>>
>> Hi *,
>>
>> this is not new to those of you who attended my talk on the F2F 2016 in
>> Brno. Let me explain the idea here again for all others who did not have
>> a chance to be there.
>>
>> Srcdeps [1] is a tool to build Maven dependencies from their sources.
>> With srcdeps, wildfly-core can depend on a specific commit of, e.g.,
>> undertow:
>>
>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>
>> where aabbccd is the git commit id to build when any undertow artifact
>> is requested during the build of wildfly-core.
>>
>> [1] describes in detail, how it works.
>>
>> The main advantage of srcdeps is that changes in components can be
>> integrated and tested in wildfly-core immediately after they are
>> committed to a public component branch. There is no need to wait for the
>> component release.
>>
>> Here in the WildFly family of projects, it is often the case that
>> something needs to be fixed in a component, but the verification (using
>> bug reproducer, or integration test) is possible only at the level of
>> wildfly or wildfly-core. Engineers typically work with snapshots
>> locally, but when their changes need to get shared (CI, reviews) in a
>> reproducible manner, snapshots cannot be used anymore.
>> In such situations a source dependency come in handy: it is very easy to
>> share and it is as reproducible as a Maven build from a specific commit
>> can be. All CIs and reviewers can work with it, because all source
>> dependency compilation is done under the hood by Maven.
>>
>> Developers working on changes that span over multiple interdependent git
>> repos can thus get feedback (i-tests, reviews) quickly without waiting
>> for releases of components.
>>
>> Srcdeps emerged in the Hawkular family of projects to solve exactly this
>> kind of situation and is in use there since around October 2015.
>>
>> When I said there is no need to wait for releases of components, I did
>> not mean that we can get rid of component releases altogether. Clearly,
>> we cannot, because i.a. for any tooling uninformed about how srcdeps
>> work, those source dependencies would simply be non-resolvable from
>> public Maven repositories. So, before releasing the dependent component
>> (such as wildfly-core) all its dependencies need to be released. To
>> enforce this, srcdeps is by default configured to make the release fail,
>> as long as there are source dependencies.
>>
>> I have sent a PR introducing srcdeps to wildfly-core:
>> https://github.com/wildfly/wildfly-core/pull/2122
>> To get a feeling how it works, checkout the branch, switch to e.g.
>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>> (that happens to be the commit id of the 1.4.8.Final tag)
>> and build wildfly-core as usual with "mvn clean install". You'll see in
>> the build log that undertow is being cloned to ~/.m2/srcdeps/io/undertow
>> and that it is built there. After the build, check that the
>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>> version of Undertow got installed to your local Maven repo (usually
>> ~/m2/repository/io/undertow/undertow-core )
>>
>> Are there any questions or comments?
>>
>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>
>> Thanks,
>>
>> Peter
>>
>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at 14:30 at
>> DevConf Brno.
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>

From kabir.khan at jboss.com  Thu Jan 26 06:51:04 2017
From: kabir.khan at jboss.com (Kabir Khan)
Date: Thu, 26 Jan 2017 11:51:04 +0000
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <0823070b-3179-988d-5b55-385a711f479a@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
	<265cc9d0-c8db-354b-b380-4861caf7c5d5@redhat.com>
	<E04A47DF-61A9-4EBD-AE5D-9AADD3B3FE5F@jboss.com>
	<08270852-7893-1241-f9e6-0cae171bcf0b@redhat.com>
	<0823070b-3179-988d-5b55-385a711f479a@redhat.com>
Message-ID: <4399BECB-6C1F-48E3-A48E-B53D7DCCDF61@jboss.com>


> On 26 Jan 2017, at 11:37, Peter Palaga <ppalaga at redhat.com> wrote:
> 
> On 2017-01-26 11:46, Juraci Paix?o Kr?hling wrote:
>> On 01/26/2017 11:25 AM, Kabir Khan wrote:
>>> Perhaps rather than a profile in the consuming project as Brian mentions, it should be disabled in srcdeps itself by default? Something along the lines that when its dependency resolution mechanism is hit, it fails unless -Dsrcdeps.enabled is passed in. Of course there is still a risk that someone adds that property to the pom by accident.
> 
> Not sure what would be the purpose of making srcdeps opt-in? - To allow CI and reviews, but to prevent merging?
Yes, or rather it would fail our default CI, apart from runs where the opt in happens explicitly, which will help with accidentally merging.
> Well, could you tell me first what is wrong with merging source dependencies?
Perhaps that is fine for some projects/branches, but for WildFly(-core) master we want proper released maven artifacts. I've had a quick look at http://maven.apache.org/xsd/core-extensions-1.0.0.xsd but there doesn't seem to be a way to pass in any config properties to extensions.xml, but perhaps something could be added to the yaml so you can set a policy there. It still runs the risk of getting merged though :)
> 
>>> Also how does this work for nested projects? Say we add this to WildFly full, and I want to test WildFly with to something which is brought in by wildfly-core (e.g. undertow, Elytron etc.). How would that work?
>> 
>> It works quite nicely. I remember seeing 3 levels of nesting and I'm
>> sure it could have even more.
>> 
>> It does delay the main build, though, as srcdeps builds the dependencies
>> on demand. It means that the first local build of "wildfly-core" that
>> depends on "elytron:1.2.3-src-abc123" will also build Elytron's revision
>> "abc123".
> 
> Yes, as Juca said, running a build of project A that has a source dependency B, the build of B can trigger yet another build of its own source dependency C.
> 
> Perhaps I should note that the dependency builds run with -DskipTests by default so that they finish faster. For dependencies known to use checkstyle, license plugin or similar, their builds can be configured to skip those too to run even faster, as I do e.g. here https://github.com/wildfly/wildfly-core/pull/2122/commits/65326ef4ff6abc5673a605f2394003f9a9537fdb#diff-34efcdaa51afff46bba8cd9c1387e6f5R146
> 
> Further, once a source dependency is built, it is installed to Maven local repository and re-used from there for all subsequent builds that require it. A source dependency thus makes just the first build slower.
> The local git repositories are kept too and therefore only the first clone takes long. All subsequent dependency builds use just fetch and reset.
> 
> -- P
> 



From ppalaga at redhat.com  Thu Jan 26 07:09:07 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Thu, 26 Jan 2017 13:09:07 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <4399BECB-6C1F-48E3-A48E-B53D7DCCDF61@jboss.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
	<265cc9d0-c8db-354b-b380-4861caf7c5d5@redhat.com>
	<E04A47DF-61A9-4EBD-AE5D-9AADD3B3FE5F@jboss.com>
	<08270852-7893-1241-f9e6-0cae171bcf0b@redhat.com>
	<0823070b-3179-988d-5b55-385a711f479a@redhat.com>
	<4399BECB-6C1F-48E3-A48E-B53D7DCCDF61@jboss.com>
Message-ID: <0bbd6256-e309-3a5b-76e7-89efba937116@redhat.com>

On 2017-01-26 12:51, Kabir Khan wrote:
>
>> On 26 Jan 2017, at 11:37, Peter Palaga <ppalaga at redhat.com> wrote:
>>
>> On 2017-01-26 11:46, Juraci Paix?o Kr?hling wrote:
>>> On 01/26/2017 11:25 AM, Kabir Khan wrote:
>>>> Perhaps rather than a profile in the consuming project as Brian mentions, it should be disabled in srcdeps itself by default? Something along the lines that when its dependency resolution mechanism is hit, it fails unless -Dsrcdeps.enabled is passed in. Of course there is still a risk that someone adds that property to the pom by accident.
>>
>> Not sure what would be the purpose of making srcdeps opt-in? - To allow CI and reviews, but to prevent merging?
> Yes, or rather it would fail our default CI, apart from runs where the opt in happens explicitly, which will help with accidentally merging.
>> Well, could you tell me first what is wrong with merging source dependencies?
> Perhaps that is fine for some projects/branches, but for WildFly(-core) master we want proper released maven artifacts.

Yes, I see your preference, but I do not see clearly why you want to ban 
source dependencies from master.

 > I've had a quick look at 
http://maven.apache.org/xsd/core-extensions-1.0.0.xsd but there doesn't 
seem to be a way to pass in any config properties to extensions.xml, but 
perhaps something could be added to the yaml so you can set a policy 
there. It still runs the risk of getting merged though :)

Yes, extensions.xml does not allow for passing any config params to 
extensions. Srcdeps.yaml would be a better place for such an option. I 
can add it once there is enough consensus to go that way. -- P

>>
>>>> Also how does this work for nested projects? Say we add this to WildFly full, and I want to test WildFly with to something which is brought in by wildfly-core (e.g. undertow, Elytron etc.). How would that work?
>>>
>>> It works quite nicely. I remember seeing 3 levels of nesting and I'm
>>> sure it could have even more.
>>>
>>> It does delay the main build, though, as srcdeps builds the dependencies
>>> on demand. It means that the first local build of "wildfly-core" that
>>> depends on "elytron:1.2.3-src-abc123" will also build Elytron's revision
>>> "abc123".
>>
>> Yes, as Juca said, running a build of project A that has a source dependency B, the build of B can trigger yet another build of its own source dependency C.
>>
>> Perhaps I should note that the dependency builds run with -DskipTests by default so that they finish faster. For dependencies known to use checkstyle, license plugin or similar, their builds can be configured to skip those too to run even faster, as I do e.g. here https://github.com/wildfly/wildfly-core/pull/2122/commits/65326ef4ff6abc5673a605f2394003f9a9537fdb#diff-34efcdaa51afff46bba8cd9c1387e6f5R146
>>
>> Further, once a source dependency is built, it is installed to Maven local repository and re-used from there for all subsequent builds that require it. A source dependency thus makes just the first build slower.
>> The local git repositories are kept too and therefore only the first clone takes long. All subsequent dependency builds use just fetch and reset.
>>
>> -- P
>>
>


From ppalaga at redhat.com  Thu Jan 26 07:42:20 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Thu, 26 Jan 2017 13:42:20 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <cb3c9ece-98db-e1ba-af4f-738968d419af@jboss.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<cb3c9ece-98db-e1ba-af4f-738968d419af@jboss.com>
Message-ID: <f0fdcf89-07f7-2c2d-7fbb-a2c1155e0872@redhat.com>

Thanks for the comments, Darran, more inline...

On 2017-01-26 12:40, Darran Lofthouse wrote:
> On 26/01/17 01:02, Brian Stansberry wrote:
>> My only concerns with this would relate to comitting this kind of src dependency to the poms in the main branches in the widlfly/wildfly and wildfly/wildfly-core repos. We?ve managed to survive up to now with little or no need for that kind of thing, so until we get used to using this in other ways IMHO we should follow the KISS principle and forbid that.
>
> I like the idea of this being available for teams working on a topic
> branch but also think we should avoid this kind of dependency in master
> - engineers are continually running builds so I don't think we should
> add additional source checkouts and builds.

So you also vote for having the support for source dependencies when the 
e.g. the wildfly-core CI builds a PR that integrates a change in Elytron 
or when a reviewer does the same, but such PRs should wait for a proper 
release of the component, right?

> I know Elytron has caused huge amounts of instability but generally I
> think we want these branches in a state where we could tag if we needed to.

"these branches" - you mean topic branches in a component (such as 
Elytron) or in a consuming project (such as WF Core) or both?

> When these downloaded builds are executed are the test cases within also
> executed?

The builds of source dependencies run with -DskipTests by default so 
that they finish faster. An explicit per git repository setting in 
srcdeps.yaml is needed if somebody wants a dependency build to run 
without -DskipTests.

>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with a SRC dependency to get CI or review and accidentally gets merged. But I suppose that?s not the end of the world, so long as the release process will eventually detect it and fail.
>
> Merging topic branches these temporary SHAs would be in the history but
> maybe the PR CI runs could verify they are not present in the final merge.
>
>> Can making srcdeps fail (or just disabling it) be turned on via a maven profile? With that we could set up such a profile and turn it on in CI jobs that are testing branches where it?s forbidden (e.g. the nightly builds of master.)
>>
>> Oh, one other concern ? how robust is this in the face of poor maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml. If that gets out of date or something is the only effect that using a src dependency for the affected item doesn't work?
>
> I think that file would be better almost empty

The size of the srcdeps.yaml file depends on how many dependencies the 
given project has and which of those are selected for the srcdeps 
support. In the PR https://git.io/vMjUC , I have included only the 
projects owned by us, excluding non-Maven and non-git projects. I am 
open to discussion whether we should include or exclude more projects.

> but not hidden.

The .mvn directory was introduced by Maven 3.3.1. Maven reads 
extensions.xml file from there and I found it rather natural to have 
srcdeps.yaml file there too.

> Thinking about the topic branch development, within the Elytron team we
> are using some SNAPSHOTs from the official repos and at times we have
> forked projects into our incubator and developed against SNAPSHOTs from
> those

Allowing SNAPSHOTs on any remote Maven repository leads to loosing 
reproducibility of the builds of the dependent project. Srcdeps offers 
much more reproducibility and portalbility over environments in such 
situations.

> as as you select the SHA for the build you would also need to
> select which repository you really want.

Yes, the mapping which dependencies are built from which git 
repositories is defined in srcdeps.yaml. I see no problem in having one 
git URL in one branch and having different git URL in another branch.

Thanks,

Peter

>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com> wrote:
>>>
>>> Hi *,
>>>
>>> this is not new to those of you who attended my talk on the F2F 2016 in
>>> Brno. Let me explain the idea here again for all others who did not have
>>> a chance to be there.
>>>
>>> Srcdeps [1] is a tool to build Maven dependencies from their sources.
>>> With srcdeps, wildfly-core can depend on a specific commit of, e.g.,
>>> undertow:
>>>
>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>
>>> where aabbccd is the git commit id to build when any undertow artifact
>>> is requested during the build of wildfly-core.
>>>
>>> [1] describes in detail, how it works.
>>>
>>> The main advantage of srcdeps is that changes in components can be
>>> integrated and tested in wildfly-core immediately after they are
>>> committed to a public component branch. There is no need to wait for the
>>> component release.
>>>
>>> Here in the WildFly family of projects, it is often the case that
>>> something needs to be fixed in a component, but the verification (using
>>> bug reproducer, or integration test) is possible only at the level of
>>> wildfly or wildfly-core. Engineers typically work with snapshots
>>> locally, but when their changes need to get shared (CI, reviews) in a
>>> reproducible manner, snapshots cannot be used anymore.
>>> In such situations a source dependency come in handy: it is very easy to
>>> share and it is as reproducible as a Maven build from a specific commit
>>> can be. All CIs and reviewers can work with it, because all source
>>> dependency compilation is done under the hood by Maven.
>>>
>>> Developers working on changes that span over multiple interdependent git
>>> repos can thus get feedback (i-tests, reviews) quickly without waiting
>>> for releases of components.
>>>
>>> Srcdeps emerged in the Hawkular family of projects to solve exactly this
>>> kind of situation and is in use there since around October 2015.
>>>
>>> When I said there is no need to wait for releases of components, I did
>>> not mean that we can get rid of component releases altogether. Clearly,
>>> we cannot, because i.a. for any tooling uninformed about how srcdeps
>>> work, those source dependencies would simply be non-resolvable from
>>> public Maven repositories. So, before releasing the dependent component
>>> (such as wildfly-core) all its dependencies need to be released. To
>>> enforce this, srcdeps is by default configured to make the release fail,
>>> as long as there are source dependencies.
>>>
>>> I have sent a PR introducing srcdeps to wildfly-core:
>>> https://github.com/wildfly/wildfly-core/pull/2122
>>> To get a feeling how it works, checkout the branch, switch to e.g.
>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>> (that happens to be the commit id of the 1.4.8.Final tag)
>>> and build wildfly-core as usual with "mvn clean install". You'll see in
>>> the build log that undertow is being cloned to ~/.m2/srcdeps/io/undertow
>>> and that it is built there. After the build, check that the
>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>> version of Undertow got installed to your local Maven repo (usually
>>> ~/m2/repository/io/undertow/undertow-core )
>>>
>>> Are there any questions or comments?
>>>
>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>
>>> Thanks,
>>>
>>> Peter
>>>
>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at 14:30 at
>>> DevConf Brno.
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


From rsvoboda at redhat.com  Thu Jan 26 07:46:26 2017
From: rsvoboda at redhat.com (Rostislav Svoboda)
Date: Thu, 26 Jan 2017 07:46:26 -0500 (EST)
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <0bbd6256-e309-3a5b-76e7-89efba937116@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
	<265cc9d0-c8db-354b-b380-4861caf7c5d5@redhat.com>
	<E04A47DF-61A9-4EBD-AE5D-9AADD3B3FE5F@jboss.com>
	<08270852-7893-1241-f9e6-0cae171bcf0b@redhat.com>
	<0823070b-3179-988d-5b55-385a711f479a@redhat.com>
	<4399BECB-6C1F-48E3-A48E-B53D7DCCDF61@jboss.com>
	<0bbd6256-e309-3a5b-76e7-89efba937116@redhat.com>
Message-ID: <591104211.1283335.1485434786472.JavaMail.zimbra@redhat.com>

> On 2017-01-26 12:51, Kabir Khan wrote:
> >
> >> On 26 Jan 2017, at 11:37, Peter Palaga <ppalaga at redhat.com> wrote:
> >>
> >> On 2017-01-26 11:46, Juraci Paix?o Kr?hling wrote:
> >>> On 01/26/2017 11:25 AM, Kabir Khan wrote:
> >>>> Perhaps rather than a profile in the consuming project as Brian
> >>>> mentions, it should be disabled in srcdeps itself by default? Something
> >>>> along the lines that when its dependency resolution mechanism is hit,
> >>>> it fails unless -Dsrcdeps.enabled is passed in. Of course there is
> >>>> still a risk that someone adds that property to the pom by accident.
> >>
> >> Not sure what would be the purpose of making srcdeps opt-in? - To allow CI
> >> and reviews, but to prevent merging?
> > Yes, or rather it would fail our default CI, apart from runs where the opt
> > in happens explicitly, which will help with accidentally merging.
> >> Well, could you tell me first what is wrong with merging source
> >> dependencies?
> > Perhaps that is fine for some projects/branches, but for WildFly(-core)
> > master we want proper released maven artifacts.
> 
> Yes, I see your preference, but I do not see clearly why you want to ban
> source dependencies from master.


I think the question is different:

Which of current problems is srcdeps solving ?

Rostislav

>  > I've had a quick look at
> http://maven.apache.org/xsd/core-extensions-1.0.0.xsd but there doesn't
> seem to be a way to pass in any config properties to extensions.xml, but
> perhaps something could be added to the yaml so you can set a policy
> there. It still runs the risk of getting merged though :)
> 
> Yes, extensions.xml does not allow for passing any config params to
> extensions. Srcdeps.yaml would be a better place for such an option. I
> can add it once there is enough consensus to go that way. -- P
> 
> >>
> >>>> Also how does this work for nested projects? Say we add this to WildFly
> >>>> full, and I want to test WildFly with to something which is brought in
> >>>> by wildfly-core (e.g. undertow, Elytron etc.). How would that work?
> >>>
> >>> It works quite nicely. I remember seeing 3 levels of nesting and I'm
> >>> sure it could have even more.
> >>>
> >>> It does delay the main build, though, as srcdeps builds the dependencies
> >>> on demand. It means that the first local build of "wildfly-core" that
> >>> depends on "elytron:1.2.3-src-abc123" will also build Elytron's revision
> >>> "abc123".
> >>
> >> Yes, as Juca said, running a build of project A that has a source
> >> dependency B, the build of B can trigger yet another build of its own
> >> source dependency C.
> >>
> >> Perhaps I should note that the dependency builds run with -DskipTests by
> >> default so that they finish faster. For dependencies known to use
> >> checkstyle, license plugin or similar, their builds can be configured to
> >> skip those too to run even faster, as I do e.g. here
> >> https://github.com/wildfly/wildfly-core/pull/2122/commits/65326ef4ff6abc5673a605f2394003f9a9537fdb#diff-34efcdaa51afff46bba8cd9c1387e6f5R146
> >>
> >> Further, once a source dependency is built, it is installed to Maven local
> >> repository and re-used from there for all subsequent builds that require
> >> it. A source dependency thus makes just the first build slower.
> >> The local git repositories are kept too and therefore only the first clone
> >> takes long. All subsequent dependency builds use just fetch and reset.
> >>
> >> -- P
> >>
> >
> 
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


From darran.lofthouse at jboss.com  Thu Jan 26 07:56:39 2017
From: darran.lofthouse at jboss.com (Darran Lofthouse)
Date: Thu, 26 Jan 2017 12:56:39 +0000
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <f0fdcf89-07f7-2c2d-7fbb-a2c1155e0872@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<cb3c9ece-98db-e1ba-af4f-738968d419af@jboss.com>
	<f0fdcf89-07f7-2c2d-7fbb-a2c1155e0872@redhat.com>
Message-ID: <bd68568d-c871-0b40-54fa-0a7d0a1cd44e@jboss.com>



On 26/01/17 12:42, Peter Palaga wrote:
> Thanks for the comments, Darran, more inline...
>
> On 2017-01-26 12:40, Darran Lofthouse wrote:
>> On 26/01/17 01:02, Brian Stansberry wrote:
>>> My only concerns with this would relate to comitting this kind of src
>>> dependency to the poms in the main branches in the widlfly/wildfly
>>> and wildfly/wildfly-core repos. We?ve managed to survive up to now
>>> with little or no need for that kind of thing, so until we get used
>>> to using this in other ways IMHO we should follow the KISS principle
>>> and forbid that.
>>
>> I like the idea of this being available for teams working on a topic
>> branch but also think we should avoid this kind of dependency in master
>> - engineers are continually running builds so I don't think we should
>> add additional source checkouts and builds.
>
> So you also vote for having the support for source dependencies when the
> e.g. the wildfly-core CI builds a PR that integrates a change in Elytron
> or when a reviewer does the same, but such PRs should wait for a proper
> release of the component, right?

I think by the time we get to the PR stage engineers should have had an 
opportunity to discuss the proposed solution sufficiently that we 
shouldn't need a srcdep and should have been able to tag Elytron.

>> I know Elytron has caused huge amounts of instability but generally I
>> think we want these branches in a state where we could tag if we
>> needed to.
>
> "these branches" - you mean topic branches in a component (such as
> Elytron) or in a consuming project (such as WF Core) or both?

I think WildFly Core / master and WildFly / master are generally not 
supposed to be as unstable as we have made them recently.

>> When these downloaded builds are executed are the test cases within also
>> executed?
>
> The builds of source dependencies run with -DskipTests by default so
> that they finish faster. An explicit per git repository setting in
> srcdeps.yaml is needed if somebody wants a dependency build to run
> without -DskipTests.
>
>>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>>> a SRC dependency to get CI or review and accidentally gets merged.
>>> But I suppose that?s not the end of the world, so long as the release
>>> process will eventually detect it and fail.
>>
>> Merging topic branches these temporary SHAs would be in the history but
>> maybe the PR CI runs could verify they are not present in the final
>> merge.
>>
>>> Can making srcdeps fail (or just disabling it) be turned on via a
>>> maven profile? With that we could set up such a profile and turn it
>>> on in CI jobs that are testing branches where it?s forbidden (e.g.
>>> the nightly builds of master.)
>>>
>>> Oh, one other concern ? how robust is this in the face of poor
>>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml. If
>>> that gets out of date or something is the only effect that using a
>>> src dependency for the affected item doesn't work?
>>
>> I think that file would be better almost empty
>
> The size of the srcdeps.yaml file depends on how many dependencies the
> given project has and which of those are selected for the srcdeps
> support. In the PR https://git.io/vMjUC , I have included only the
> projects owned by us, excluding non-Maven and non-git projects. I am
> open to discussion whether we should include or exclude more projects.
>
>> but not hidden.
>
> The .mvn directory was introduced by Maven 3.3.1. Maven reads
> extensions.xml file from there and I found it rather natural to have
> srcdeps.yaml file there too.

What that means is that a file that can directly influence the outcome 
of a local build is hidden from the engineer.  There are already quite a 
few things in out builds where when they go wrong we have to spend time 
tracking them down, having a hidden configuration may not help this.

>> Thinking about the topic branch development, within the Elytron team we
>> are using some SNAPSHOTs from the official repos and at times we have
>> forked projects into our incubator and developed against SNAPSHOTs from
>> those
>
> Allowing SNAPSHOTs on any remote Maven repository leads to loosing
> reproducibility of the builds of the dependent project. Srcdeps offers
> much more reproducibility and portalbility over environments in such
> situations.

Yes I am saying in the development of the Elytron topic branches this 
could help a lot - but the assumptions as to which git repos we are 
using would be wrong.

>> as as you select the SHA for the build you would also need to
>> select which repository you really want.
>
> Yes, the mapping which dependencies are built from which git
> repositories is defined in srcdeps.yaml. I see no problem in having one
> git URL in one branch and having different git URL in another branch.

Which I think as this is a file engineers could be editing to tweak 
their builds means it should not be hidden.

> Thanks,
>
> Peter
>
>>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com> wrote:
>>>>
>>>> Hi *,
>>>>
>>>> this is not new to those of you who attended my talk on the F2F 2016 in
>>>> Brno. Let me explain the idea here again for all others who did not
>>>> have
>>>> a chance to be there.
>>>>
>>>> Srcdeps [1] is a tool to build Maven dependencies from their sources.
>>>> With srcdeps, wildfly-core can depend on a specific commit of, e.g.,
>>>> undertow:
>>>>
>>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>>
>>>>
>>>> where aabbccd is the git commit id to build when any undertow artifact
>>>> is requested during the build of wildfly-core.
>>>>
>>>> [1] describes in detail, how it works.
>>>>
>>>> The main advantage of srcdeps is that changes in components can be
>>>> integrated and tested in wildfly-core immediately after they are
>>>> committed to a public component branch. There is no need to wait for
>>>> the
>>>> component release.
>>>>
>>>> Here in the WildFly family of projects, it is often the case that
>>>> something needs to be fixed in a component, but the verification (using
>>>> bug reproducer, or integration test) is possible only at the level of
>>>> wildfly or wildfly-core. Engineers typically work with snapshots
>>>> locally, but when their changes need to get shared (CI, reviews) in a
>>>> reproducible manner, snapshots cannot be used anymore.
>>>> In such situations a source dependency come in handy: it is very
>>>> easy to
>>>> share and it is as reproducible as a Maven build from a specific commit
>>>> can be. All CIs and reviewers can work with it, because all source
>>>> dependency compilation is done under the hood by Maven.
>>>>
>>>> Developers working on changes that span over multiple interdependent
>>>> git
>>>> repos can thus get feedback (i-tests, reviews) quickly without waiting
>>>> for releases of components.
>>>>
>>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>>> this
>>>> kind of situation and is in use there since around October 2015.
>>>>
>>>> When I said there is no need to wait for releases of components, I did
>>>> not mean that we can get rid of component releases altogether. Clearly,
>>>> we cannot, because i.a. for any tooling uninformed about how srcdeps
>>>> work, those source dependencies would simply be non-resolvable from
>>>> public Maven repositories. So, before releasing the dependent component
>>>> (such as wildfly-core) all its dependencies need to be released. To
>>>> enforce this, srcdeps is by default configured to make the release
>>>> fail,
>>>> as long as there are source dependencies.
>>>>
>>>> I have sent a PR introducing srcdeps to wildfly-core:
>>>> https://github.com/wildfly/wildfly-core/pull/2122
>>>> To get a feeling how it works, checkout the branch, switch to e.g.
>>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>>>
>>>> (that happens to be the commit id of the 1.4.8.Final tag)
>>>> and build wildfly-core as usual with "mvn clean install". You'll see in
>>>> the build log that undertow is being cloned to
>>>> ~/.m2/srcdeps/io/undertow
>>>> and that it is built there. After the build, check that the
>>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>>> version of Undertow got installed to your local Maven repo (usually
>>>> ~/m2/repository/io/undertow/undertow-core )
>>>>
>>>> Are there any questions or comments?
>>>>
>>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>>
>>>> Thanks,
>>>>
>>>> Peter
>>>>
>>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>>> 14:30 at
>>>> DevConf Brno.
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> wildfly-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>

From ppalaga at redhat.com  Thu Jan 26 08:33:51 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Thu, 26 Jan 2017 14:33:51 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <591104211.1283335.1485434786472.JavaMail.zimbra@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<fdaa91a7-d363-0e67-7720-8c44b6d55d08@redhat.com>
	<265cc9d0-c8db-354b-b380-4861caf7c5d5@redhat.com>
	<E04A47DF-61A9-4EBD-AE5D-9AADD3B3FE5F@jboss.com>
	<08270852-7893-1241-f9e6-0cae171bcf0b@redhat.com>
	<0823070b-3179-988d-5b55-385a711f479a@redhat.com>
	<4399BECB-6C1F-48E3-A48E-B53D7DCCDF61@jboss.com>
	<0bbd6256-e309-3a5b-76e7-89efba937116@redhat.com>
	<591104211.1283335.1485434786472.JavaMail.zimbra@redhat.com>
Message-ID: <d1562663-b98f-42c0-8ab0-c1f0411223a9@redhat.com>

On 2017-01-26 13:46, Rostislav Svoboda wrote:
>> On 2017-01-26 12:51, Kabir Khan wrote:
>>>
>>>> On 26 Jan 2017, at 11:37, Peter Palaga <ppalaga at redhat.com> wrote:
>>>>
>>>> On 2017-01-26 11:46, Juraci Paix?o Kr?hling wrote:
>>>>> On 01/26/2017 11:25 AM, Kabir Khan wrote:
>>>>>> Perhaps rather than a profile in the consuming project as Brian
>>>>>> mentions, it should be disabled in srcdeps itself by default? Something
>>>>>> along the lines that when its dependency resolution mechanism is hit,
>>>>>> it fails unless -Dsrcdeps.enabled is passed in. Of course there is
>>>>>> still a risk that someone adds that property to the pom by accident.
>>>>
>>>> Not sure what would be the purpose of making srcdeps opt-in? - To allow CI
>>>> and reviews, but to prevent merging?
>>> Yes, or rather it would fail our default CI, apart from runs where the opt
>>> in happens explicitly, which will help with accidentally merging.
>>>> Well, could you tell me first what is wrong with merging source
>>>> dependencies?
>>> Perhaps that is fine for some projects/branches, but for WildFly(-core)
>>> master we want proper released maven artifacts.
>>
>> Yes, I see your preference, but I do not see clearly why you want to ban
>> source dependencies from master.
>
>
> I think the question is different:
>
> Which of current problems is srcdeps solving ?

It is an improvement of the development process through simplifying it 
and making it faster.
As I said in the initial message, the main advantage of srcdeps is that 
changes in components can be integrated, tested and reviewed in wildfly 
and wildfly-core immediately after they are committed to a public 
component branch. The developer does not need to wait for the component 
release. He gets the feedback from CI and reviewers faster. I do not 
think anything changes for quality engineers, given that the current 
proposal assumes srcdeps-free releases. -- P

> Rostislav
>
>>  > I've had a quick look at
>> http://maven.apache.org/xsd/core-extensions-1.0.0.xsd but there doesn't
>> seem to be a way to pass in any config properties to extensions.xml, but
>> perhaps something could be added to the yaml so you can set a policy
>> there. It still runs the risk of getting merged though :)
>>
>> Yes, extensions.xml does not allow for passing any config params to
>> extensions. Srcdeps.yaml would be a better place for such an option. I
>> can add it once there is enough consensus to go that way. -- P
>>
>>>>
>>>>>> Also how does this work for nested projects? Say we add this to WildFly
>>>>>> full, and I want to test WildFly with to something which is brought in
>>>>>> by wildfly-core (e.g. undertow, Elytron etc.). How would that work?
>>>>>
>>>>> It works quite nicely. I remember seeing 3 levels of nesting and I'm
>>>>> sure it could have even more.
>>>>>
>>>>> It does delay the main build, though, as srcdeps builds the dependencies
>>>>> on demand. It means that the first local build of "wildfly-core" that
>>>>> depends on "elytron:1.2.3-src-abc123" will also build Elytron's revision
>>>>> "abc123".
>>>>
>>>> Yes, as Juca said, running a build of project A that has a source
>>>> dependency B, the build of B can trigger yet another build of its own
>>>> source dependency C.
>>>>
>>>> Perhaps I should note that the dependency builds run with -DskipTests by
>>>> default so that they finish faster. For dependencies known to use
>>>> checkstyle, license plugin or similar, their builds can be configured to
>>>> skip those too to run even faster, as I do e.g. here
>>>> https://github.com/wildfly/wildfly-core/pull/2122/commits/65326ef4ff6abc5673a605f2394003f9a9537fdb#diff-34efcdaa51afff46bba8cd9c1387e6f5R146
>>>>
>>>> Further, once a source dependency is built, it is installed to Maven local
>>>> repository and re-used from there for all subsequent builds that require
>>>> it. A source dependency thus makes just the first build slower.
>>>> The local git repositories are kept too and therefore only the first clone
>>>> takes long. All subsequent dependency builds use just fetch and reset.
>>>>
>>>> -- P
>>>>
>>>
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev


From brian.stansberry at redhat.com  Thu Jan 26 09:44:48 2017
From: brian.stansberry at redhat.com (Brian Stansberry)
Date: Thu, 26 Jan 2017 08:44:48 -0600
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
Message-ID: <1EC27C97-CC18-4CB6-A434-2E053A8999CE@redhat.com>

There?s been a lot of discussion overnight, but I?ll reply to this one directly since my answers better align with your questions here. :)

> On Jan 26, 2017, at 2:54 AM, Peter Palaga <ppalaga at redhat.com> wrote:
> 
> Hi Brian, thanks for your comments, more inline...
> 
> On 2017-01-26 02:02, Brian Stansberry wrote:
>> My only concerns with this would relate to comitting this kind of src
>> dependency to the poms in the main branches in the widlfly/wildfly
>> and wildfly/wildfly-core repos. We?ve managed to survive up to now
>> with little or no need for that kind of thing, so until we get used
>> to using this in other ways IMHO we should follow the KISS principle
>> and forbid that.
> 
> Maybe I overestimate the amount of changes that span over multiple git repos. Maybe you in the Core team do not do this often. But for us in the Sustaining Engineering Team, this is quite a typical situation. A substantial part of the reports from customers come with a description how to reproduce on the whole server, but they need to be fixed in a component. Having srcdeps would make the CP process simpler and faster, allowing us to uncover the conflicts and regressions earlier.

I don?t see how merging to the main branches is required to get this benefit. Git topic branches are fully sharable and CI jobs against them are easily done. All CI tests of pull requests are tests of topic branches.

But, in any case perhaps you?ve seen clear need for merging to the main branches with the EAP CP branches. I haven?t seen it in WildFly / WildFly Core. I deliberately used specific repo names in my last comment to try and scope it. ;)

Note I?m not saying we should disallow PRs with src deps in the pom. We should just disallow merging until those are replaced.

> 
>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>> a SRC dependency to get CI or review and accidentally gets merged.
> 
> Oh, I am just realizing I have not said anything about merging. I actually do want to propose that commits with source dependencies get merged to e.g. wildfly-core master as early as possible. Those are the key points of Continuous Integration: get feedback quickly, and merge as soon as possible. This is exactly what Hawkular is doing since more than a year.

We regularly produce releases (ideally weekly for WildFly Core), often at short notice under pressure.  Allowing merging of changes that are not acceptable for release increases the risk and effort required to do that, since now we have to scan for src deps and figure out how to get them out of the build. Perhaps needing assistance from whoever added the src dep and the lead of relevant component, both of whom are on the other side of the world asleep. (This is a real issue since we often do releases on Friday afternoon US time or Monday morning European time.) We already have too much risk and effort doing releases so adding more will need a really strong justification.

> 
>> But I suppose that?s not the end of the world, so long as the release
>> process will eventually detect it and fail.
> 
> Yes, source dependencies on a stable branch do not harm. They just need to be avoided in releases (for which srcdeps offers technical means).

They do do harm as they mean the branch is no longer releasable. It?s not end-of-the-world harm but it?s harm.

> 
>> Can making srcdeps fail (or just disabling it) be turned on via a
>> maven profile? With that we could set up such a profile and turn it
>> on in CI jobs that are testing branches where it?s forbidden (e.g.
>> the nightly builds of master.)
> 
> Yes, the feature is called "failWith profiles" and can be configured in .mvn/srcdeps.yaml, like here in this srcdeps quickstart: https://github.com/srcdeps/srcdeps-maven/blob/master/srcdeps-maven-quickstarts/srcdeps-mvn-git-profile-quickstart/.mvn/srcdeps.yaml#L33 
> There is also "failWith properties" and "failWith goals". It is documented here: https://github.com/srcdeps/srcdeps-core/blob/master/doc/srcdeps.yaml#L130
> By default there is failWith: {goals: release:prepare, release:perform}. Projects that do not use the release plugin can set e.g. failWith: {goals: deploy:deploy} or whatever else distinguishes their releases.
> 

Thanks.

>> Oh, one other concern ? how robust is this in the face of poor
>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml.
> 
> Which parts are boilerpate?

All of it. :)

I?m not using that word as an attack. I?m just saying it?s extra text that needs to be maintained, and since it?s separate from the usual place similar text occurs (the poms) it is more likely to diverge.

> 
>> If
>> that gets out of date or something is the only effect that using a
>> src dependency for the affected item doesn't work?
> 
> Yes, I think so. As long as the .mvn/srcdeps.yaml file is syntactically correct, any misconfiguration there should not have any other effect than eventually breaking an embedded build.
> 
> Generally, the things configured in .mvn/srcdeps.yaml tend to be quite stable - it is basically just mapping from GAVs to their respective git URLs. Git URLs do not change often. It is true that dependency artifacts come and go, but as long as their groupIds are selected reasonably (one groupId occurs in not more than one git repo) the mapping itself can be quite stable over time too.

Yeah, that?s true. Where this file would be more likely to go unmaintained is adding new entries or cleaning out old ones. But the latter is just noise and if the only harm of the former is a srcdep can?t be used for that lib, then that will naturally get handled by whoever wants to use the srcdep.

> 
> Thanks,
> 
> Peter
> 
>> 
>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com>
>>> wrote:
>>> 
>>> Hi *,
>>> 
>>> this is not new to those of you who attended my talk on the F2F
>>> 2016 in Brno. Let me explain the idea here again for all others who
>>> did not have a chance to be there.
>>> 
>>> Srcdeps [1] is a tool to build Maven dependencies from their
>>> sources. With srcdeps, wildfly-core can depend on a specific commit
>>> of, e.g., undertow:
>>> 
>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>> 
>>> 
>>> 
> where aabbccd is the git commit id to build when any undertow artifact
>>> is requested during the build of wildfly-core.
>>> 
>>> [1] describes in detail, how it works.
>>> 
>>> The main advantage of srcdeps is that changes in components can be
>>> integrated and tested in wildfly-core immediately after they are
>>> committed to a public component branch. There is no need to wait
>>> for the component release.
>>> 
>>> Here in the WildFly family of projects, it is often the case that
>>> something needs to be fixed in a component, but the verification
>>> (using bug reproducer, or integration test) is possible only at the
>>> level of wildfly or wildfly-core. Engineers typically work with
>>> snapshots locally, but when their changes need to get shared (CI,
>>> reviews) in a reproducible manner, snapshots cannot be used
>>> anymore. In such situations a source dependency come in handy: it
>>> is very easy to share and it is as reproducible as a Maven build
>>> from a specific commit can be. All CIs and reviewers can work with
>>> it, because all source dependency compilation is done under the
>>> hood by Maven.
>>> 
>>> Developers working on changes that span over multiple
>>> interdependent git repos can thus get feedback (i-tests, reviews)
>>> quickly without waiting for releases of components.
>>> 
>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>> this kind of situation and is in use there since around October
>>> 2015.
>>> 
>>> When I said there is no need to wait for releases of components, I
>>> did not mean that we can get rid of component releases altogether.
>>> Clearly, we cannot, because i.a. for any tooling uninformed about
>>> how srcdeps work, those source dependencies would simply be
>>> non-resolvable from public Maven repositories. So, before releasing
>>> the dependent component (such as wildfly-core) all its dependencies
>>> need to be released. To enforce this, srcdeps is by default
>>> configured to make the release fail, as long as there are source
>>> dependencies.
>>> 
>>> I have sent a PR introducing srcdeps to wildfly-core:
>>> https://github.com/wildfly/wildfly-core/pull/2122 To get a feeling
>>> how it works, checkout the branch, switch to e.g.
>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>> (that happens to be the commit id of the 1.4.8.Final tag) and
>>> build wildfly-core as usual with "mvn clean install". You'll see in
>>> the build log that undertow is being cloned to
>>> ~/.m2/srcdeps/io/undertow and that it is built there. After the
>>> build, check that the
>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>> version of Undertow got installed to your local Maven repo (usually
>>> ~/m2/repository/io/undertow/undertow-core )
>>> 
>>> Are there any questions or comments?
>>> 
>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>> 
>>> Thanks,
>>> 
>>> Peter
>>> 
>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>> 14:30 at DevConf Brno.
>>> _______________________________________________ wildfly-dev mailing
>>> list wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>> 
> 

-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat





From ppalaga at redhat.com  Thu Jan 26 11:28:06 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Thu, 26 Jan 2017 17:28:06 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <1EC27C97-CC18-4CB6-A434-2E053A8999CE@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<1EC27C97-CC18-4CB6-A434-2E053A8999CE@redhat.com>
Message-ID: <60823934-405c-c800-074e-86f6da854e32@redhat.com>

On 2017-01-26 15:44, Brian Stansberry wrote:
> There?s been a lot of discussion overnight, but I?ll reply to this one directly since my answers better align with your questions here. :)
>
>> On Jan 26, 2017, at 2:54 AM, Peter Palaga <ppalaga at redhat.com> wrote:
>>
>> Hi Brian, thanks for your comments, more inline...
>>
>> On 2017-01-26 02:02, Brian Stansberry wrote:
>>> My only concerns with this would relate to comitting this kind of src
>>> dependency to the poms in the main branches in the widlfly/wildfly
>>> and wildfly/wildfly-core repos. We?ve managed to survive up to now
>>> with little or no need for that kind of thing, so until we get used
>>> to using this in other ways IMHO we should follow the KISS principle
>>> and forbid that.
>>
>> Maybe I overestimate the amount of changes that span over multiple git repos. Maybe you in the Core team do not do this often. But for us in the Sustaining Engineering Team, this is quite a typical situation. A substantial part of the reports from customers come with a description how to reproduce on the whole server, but they need to be fixed in a component. Having srcdeps would make the CP process simpler and faster, allowing us to uncover the conflicts and regressions earlier.
>
> I don?t see how merging to the main branches is required to get this benefit. Git topic branches are fully sharable and CI jobs against them are easily done. All CI tests of pull requests are tests of topic branches.

Yes, for me as the submitter of the PR, it is nice to get the feedback 
from the CI and a review early, even before the component is released, 
but it is quite bothersome to have to revisit the PR again once the 
component gets released and rebase (in case there there are conflicts) 
and either upgrade to the released component version or remove the 
upgrade change (if the upgrade was merged separately).

As long as my PR is not merged, my changes are not binding for the rest 
of the team. I want my PR to get merged as fast as possible and make 
others care that their changes are compatible with mine. I want to 
happily forget about the PR as soon as possible and pick a new task :)

> But, in any case perhaps you?ve seen clear need for merging to the main branches with the EAP CP branches. I haven?t seen it in WildFly / WildFly Core. I deliberately used specific repo names in my last comment to try and scope it. ;)

My reasons for merging there in EAP CP branches are the same as here in 
the community branches: it is better for PR submitters to merge as early 
as possible to avoid conflicts, subsequent PR edits and to keep the list 
of open tasks short.

> Note I?m not saying we should disallow PRs with src deps in the pom. We should just disallow merging until those are replaced.

Yes, I understand that and I appreciate that. That would be a progress too.

>>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>>> a SRC dependency to get CI or review and accidentally gets merged.
>>
>> Oh, I am just realizing I have not said anything about merging. I actually do want to propose that commits with source dependencies get merged to e.g. wildfly-core master as early as possible. Those are the key points of Continuous Integration: get feedback quickly, and merge as soon as possible. This is exactly what Hawkular is doing since more than a year.
>
> We regularly produce releases (ideally weekly for WildFly Core), often at short notice under pressure.  Allowing merging of changes that are not acceptable for release increases the risk and effort required to do that, since now we have to scan for src deps and figure out how to get them out of the build. Perhaps needing assistance from whoever added the src dep and the lead of relevant component, both of whom are on the other side of the world asleep. (This is a real issue since we often do releases on Friday afternoon US time or Monday morning European time.) We already have too much risk and effort doing releases so adding more will need a really strong justification.

This sounds as a valid concern. I must admit I know little about how you 
plan and perform the releases of wildfly-core, wildfly and of the 
components in the community. Knowing how complex the graph of WF 
components is, I am far from underestimating any manual release efforts 
or efforts to setup a CI jobs to do that automagically. I'll have to 
gather more info about how you work.

>>> But I suppose that?s not the end of the world, so long as the release
>>> process will eventually detect it and fail.
>>
>> Yes, source dependencies on a stable branch do not harm. They just need to be avoided in releases (for which srcdeps offers technical means).
>
> They do do harm as they mean the branch is no longer releasable. It?s not end-of-the-world harm but it?s harm.

Well, I naivelly thought, that the components are obligated to provide a 
release, say, one day before a planned wildfly-core release and send a 
PRs that would then sweep out all source dependencies. And TBH, I did 
not think "releasable at any time" is important in wildfly-core. 
"Releasable once a week" still sounds good enough to me :)

>>> Can making srcdeps fail (or just disabling it) be turned on via a
>>> maven profile? With that we could set up such a profile and turn it
>>> on in CI jobs that are testing branches where it?s forbidden (e.g.
>>> the nightly builds of master.)
>>
>> Yes, the feature is called "failWith profiles" and can be configured in .mvn/srcdeps.yaml, like here in this srcdeps quickstart: https://github.com/srcdeps/srcdeps-maven/blob/master/srcdeps-maven-quickstarts/srcdeps-mvn-git-profile-quickstart/.mvn/srcdeps.yaml#L33
>> There is also "failWith properties" and "failWith goals". It is documented here: https://github.com/srcdeps/srcdeps-core/blob/master/doc/srcdeps.yaml#L130
>> By default there is failWith: {goals: release:prepare, release:perform}. Projects that do not use the release plugin can set e.g. failWith: {goals: deploy:deploy} or whatever else distinguishes their releases.
>>
>
> Thanks.
>
>>> Oh, one other concern ? how robust is this in the face of poor
>>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml.
>>
>> Which parts are boilerpate?
>
> All of it. :)
>
> I?m not using that word as an attack. I?m just saying it?s extra text that needs to be maintained, and since it?s separate from the usual place similar text occurs (the poms) it is more likely to diverge.

OK, now I know what you mean :) You are right that poms can diverge from 
srcdeps.yaml.

>>> If
>>> that gets out of date or something is the only effect that using a
>>> src dependency for the affected item doesn't work?
>>
>> Yes, I think so. As long as the .mvn/srcdeps.yaml file is syntactically correct, any misconfiguration there should not have any other effect than eventually breaking an embedded build.
>>
>> Generally, the things configured in .mvn/srcdeps.yaml tend to be quite stable - it is basically just mapping from GAVs to their respective git URLs. Git URLs do not change often. It is true that dependency artifacts come and go, but as long as their groupIds are selected reasonably (one groupId occurs in not more than one git repo) the mapping itself can be quite stable over time too.
>
> Yeah, that?s true. Where this file would be more likely to go unmaintained is adding new entries or cleaning out old ones. But the latter is just noise and if the only harm of the former is a srcdep can?t be used for that lib, then that will naturally get handled by whoever wants to use the srcdep.

Yes, exactly.

Thanks,

Peter

>>
>> Thanks,
>>
>> Peter
>>
>>>
>>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com>
>>>> wrote:
>>>>
>>>> Hi *,
>>>>
>>>> this is not new to those of you who attended my talk on the F2F
>>>> 2016 in Brno. Let me explain the idea here again for all others who
>>>> did not have a chance to be there.
>>>>
>>>> Srcdeps [1] is a tool to build Maven dependencies from their
>>>> sources. With srcdeps, wildfly-core can depend on a specific commit
>>>> of, e.g., undertow:
>>>>
>>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>>
>>>>
>>>>
>> where aabbccd is the git commit id to build when any undertow artifact
>>>> is requested during the build of wildfly-core.
>>>>
>>>> [1] describes in detail, how it works.
>>>>
>>>> The main advantage of srcdeps is that changes in components can be
>>>> integrated and tested in wildfly-core immediately after they are
>>>> committed to a public component branch. There is no need to wait
>>>> for the component release.
>>>>
>>>> Here in the WildFly family of projects, it is often the case that
>>>> something needs to be fixed in a component, but the verification
>>>> (using bug reproducer, or integration test) is possible only at the
>>>> level of wildfly or wildfly-core. Engineers typically work with
>>>> snapshots locally, but when their changes need to get shared (CI,
>>>> reviews) in a reproducible manner, snapshots cannot be used
>>>> anymore. In such situations a source dependency come in handy: it
>>>> is very easy to share and it is as reproducible as a Maven build
>>>> from a specific commit can be. All CIs and reviewers can work with
>>>> it, because all source dependency compilation is done under the
>>>> hood by Maven.
>>>>
>>>> Developers working on changes that span over multiple
>>>> interdependent git repos can thus get feedback (i-tests, reviews)
>>>> quickly without waiting for releases of components.
>>>>
>>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>>> this kind of situation and is in use there since around October
>>>> 2015.
>>>>
>>>> When I said there is no need to wait for releases of components, I
>>>> did not mean that we can get rid of component releases altogether.
>>>> Clearly, we cannot, because i.a. for any tooling uninformed about
>>>> how srcdeps work, those source dependencies would simply be
>>>> non-resolvable from public Maven repositories. So, before releasing
>>>> the dependent component (such as wildfly-core) all its dependencies
>>>> need to be released. To enforce this, srcdeps is by default
>>>> configured to make the release fail, as long as there are source
>>>> dependencies.
>>>>
>>>> I have sent a PR introducing srcdeps to wildfly-core:
>>>> https://github.com/wildfly/wildfly-core/pull/2122 To get a feeling
>>>> how it works, checkout the branch, switch to e.g.
>>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>>> (that happens to be the commit id of the 1.4.8.Final tag) and
>>>> build wildfly-core as usual with "mvn clean install". You'll see in
>>>> the build log that undertow is being cloned to
>>>> ~/.m2/srcdeps/io/undertow and that it is built there. After the
>>>> build, check that the
>>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>>> version of Undertow got installed to your local Maven repo (usually
>>>> ~/m2/repository/io/undertow/undertow-core )
>>>>
>>>> Are there any questions or comments?
>>>>
>>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>>
>>>> Thanks,
>>>>
>>>> Peter
>>>>
>>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>>> 14:30 at DevConf Brno.
>>>> _______________________________________________ wildfly-dev mailing
>>>> list wildfly-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>
>


From brian.stansberry at redhat.com  Thu Jan 26 13:11:48 2017
From: brian.stansberry at redhat.com (Brian Stansberry)
Date: Thu, 26 Jan 2017 12:11:48 -0600
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <60823934-405c-c800-074e-86f6da854e32@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<1EC27C97-CC18-4CB6-A434-2E053A8999CE@redhat.com>
	<60823934-405c-c800-074e-86f6da854e32@redhat.com>
Message-ID: <A567C0B1-5DC3-4ACD-97B6-CCA7F4194F6D@redhat.com>


> On Jan 26, 2017, at 10:28 AM, Peter Palaga <ppalaga at redhat.com> wrote:
> 
> On 2017-01-26 15:44, Brian Stansberry wrote:
>> There?s been a lot of discussion overnight, but I?ll reply to this one directly since my answers better align with your questions here. :)
>> 
>>> On Jan 26, 2017, at 2:54 AM, Peter Palaga <ppalaga at redhat.com> wrote:
>>> 
>>> Hi Brian, thanks for your comments, more inline...
>>> 
>>> On 2017-01-26 02:02, Brian Stansberry wrote:
>>>> My only concerns with this would relate to comitting this kind of src
>>>> dependency to the poms in the main branches in the widlfly/wildfly
>>>> and wildfly/wildfly-core repos. We?ve managed to survive up to now
>>>> with little or no need for that kind of thing, so until we get used
>>>> to using this in other ways IMHO we should follow the KISS principle
>>>> and forbid that.
>>> 
>>> Maybe I overestimate the amount of changes that span over multiple git repos. Maybe you in the Core team do not do this often. But for us in the Sustaining Engineering Team, this is quite a typical situation. A substantial part of the reports from customers come with a description how to reproduce on the whole server, but they need to be fixed in a component. Having srcdeps would make the CP process simpler and faster, allowing us to uncover the conflicts and regressions earlier.
>> 
>> I don?t see how merging to the main branches is required to get this benefit. Git topic branches are fully sharable and CI jobs against them are easily done. All CI tests of pull requests are tests of topic branches.
> 
> Yes, for me as the submitter of the PR, it is nice to get the feedback from the CI and a review early, even before the component is released, but it is quite bothersome to have to revisit the PR again once the component gets released and rebase (in case there there are conflicts) and either upgrade to the released component version or remove the upgrade change (if the upgrade was merged separately).
> 
> As long as my PR is not merged, my changes are not binding for the rest of the team. I want my PR to get merged as fast as possible and make others care that their changes are compatible with mine. I want to happily forget about the PR as soon as possible and pick a new task :)
> 

You?ve convinced me! Convinced me that we shouldn?t allow this. :D

We don?t have a role analogous to the ?release coordinator? used with EAP CPs, i.e. someone whose primary responsibility is coordinating to make sure that the untidy pieces get tidied. Most of our non-CR/Final releases are done as side tasks by people who are stealing time from other tasks. They need to be simple and mechanical. We also have a far greater volume of changes to manage than EAP CPs do. A process based on merging half the necessary change and then letting the issue owner walk away and assume someone else is going to come tidy up is a recipe for disaster.

>> But, in any case perhaps you?ve seen clear need for merging to the main branches with the EAP CP branches. I haven?t seen it in WildFly / WildFly Core. I deliberately used specific repo names in my last comment to try and scope it. ;)
> 
> My reasons for merging there in EAP CP branches are the same as here in the community branches: it is better for PR submitters to merge as early as possible to avoid conflicts, subsequent PR edits and to keep the list of open tasks short.
> 
>> Note I?m not saying we should disallow PRs with src deps in the pom. We should just disallow merging until those are replaced.
> 
> Yes, I understand that and I appreciate that. That would be a progress too.
> 
>>>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>>>> a SRC dependency to get CI or review and accidentally gets merged.
>>> 
>>> Oh, I am just realizing I have not said anything about merging. I actually do want to propose that commits with source dependencies get merged to e.g. wildfly-core master as early as possible. Those are the key points of Continuous Integration: get feedback quickly, and merge as soon as possible. This is exactly what Hawkular is doing since more than a year.
>> 
>> We regularly produce releases (ideally weekly for WildFly Core), often at short notice under pressure.  Allowing merging of changes that are not acceptable for release increases the risk and effort required to do that, since now we have to scan for src deps and figure out how to get them out of the build. Perhaps needing assistance from whoever added the src dep and the lead of relevant component, both of whom are on the other side of the world asleep. (This is a real issue since we often do releases on Friday afternoon US time or Monday morning European time.) We already have too much risk and effort doing releases so adding more will need a really strong justification.
> 
> This sounds as a valid concern. I must admit I know little about how you plan and perform the releases of wildfly-core, wildfly and of the components in the community. Knowing how complex the graph of WF components is, I am far from underestimating any manual release efforts or efforts to setup a CI jobs to do that automagically. I'll have to gather more info about how you work.
> 
>>>> But I suppose that?s not the end of the world, so long as the release
>>>> process will eventually detect it and fail.
>>> 
>>> Yes, source dependencies on a stable branch do not harm. They just need to be avoided in releases (for which srcdeps offers technical means).
>> 
>> They do do harm as they mean the branch is no longer releasable. It?s not end-of-the-world harm but it?s harm.
> 
> Well, I naivelly thought, that the components are obligated to provide a release, say, one day before a planned wildfly-core release and send a PRs that would then sweep out all source dependencies. And TBH, I did not think "releasable at any time" is important in wildfly-core. "Releasable once a week" still sounds good enough to me :)

Unfortunately, it?s not.

> 
>>>> Can making srcdeps fail (or just disabling it) be turned on via a
>>>> maven profile? With that we could set up such a profile and turn it
>>>> on in CI jobs that are testing branches where it?s forbidden (e.g.
>>>> the nightly builds of master.)
>>> 
>>> Yes, the feature is called "failWith profiles" and can be configured in .mvn/srcdeps.yaml, like here in this srcdeps quickstart: https://github.com/srcdeps/srcdeps-maven/blob/master/srcdeps-maven-quickstarts/srcdeps-mvn-git-profile-quickstart/.mvn/srcdeps.yaml#L33
>>> There is also "failWith properties" and "failWith goals". It is documented here: https://github.com/srcdeps/srcdeps-core/blob/master/doc/srcdeps.yaml#L130
>>> By default there is failWith: {goals: release:prepare, release:perform}. Projects that do not use the release plugin can set e.g. failWith: {goals: deploy:deploy} or whatever else distinguishes their releases.
>>> 
>> 
>> Thanks.
>> 
>>>> Oh, one other concern ? how robust is this in the face of poor
>>>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml.
>>> 
>>> Which parts are boilerpate?
>> 
>> All of it. :)
>> 
>> I?m not using that word as an attack. I?m just saying it?s extra text that needs to be maintained, and since it?s separate from the usual place similar text occurs (the poms) it is more likely to diverge.
> 
> OK, now I know what you mean :) You are right that poms can diverge from srcdeps.yaml.
> 
>>>> If
>>>> that gets out of date or something is the only effect that using a
>>>> src dependency for the affected item doesn't work?
>>> 
>>> Yes, I think so. As long as the .mvn/srcdeps.yaml file is syntactically correct, any misconfiguration there should not have any other effect than eventually breaking an embedded build.
>>> 
>>> Generally, the things configured in .mvn/srcdeps.yaml tend to be quite stable - it is basically just mapping from GAVs to their respective git URLs. Git URLs do not change often. It is true that dependency artifacts come and go, but as long as their groupIds are selected reasonably (one groupId occurs in not more than one git repo) the mapping itself can be quite stable over time too.
>> 
>> Yeah, that?s true. Where this file would be more likely to go unmaintained is adding new entries or cleaning out old ones. But the latter is just noise and if the only harm of the former is a srcdep can?t be used for that lib, then that will naturally get handled by whoever wants to use the srcdep.
> 
> Yes, exactly.
> 
> Thanks,
> 
> Peter
> 
>>> 
>>> Thanks,
>>> 
>>> Peter
>>> 
>>>> 
>>>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com>
>>>>> wrote:
>>>>> 
>>>>> Hi *,
>>>>> 
>>>>> this is not new to those of you who attended my talk on the F2F
>>>>> 2016 in Brno. Let me explain the idea here again for all others who
>>>>> did not have a chance to be there.
>>>>> 
>>>>> Srcdeps [1] is a tool to build Maven dependencies from their
>>>>> sources. With srcdeps, wildfly-core can depend on a specific commit
>>>>> of, e.g., undertow:
>>>>> 
>>>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>>> 
>>>>> 
>>>>> 
>>> where aabbccd is the git commit id to build when any undertow artifact
>>>>> is requested during the build of wildfly-core.
>>>>> 
>>>>> [1] describes in detail, how it works.
>>>>> 
>>>>> The main advantage of srcdeps is that changes in components can be
>>>>> integrated and tested in wildfly-core immediately after they are
>>>>> committed to a public component branch. There is no need to wait
>>>>> for the component release.
>>>>> 
>>>>> Here in the WildFly family of projects, it is often the case that
>>>>> something needs to be fixed in a component, but the verification
>>>>> (using bug reproducer, or integration test) is possible only at the
>>>>> level of wildfly or wildfly-core. Engineers typically work with
>>>>> snapshots locally, but when their changes need to get shared (CI,
>>>>> reviews) in a reproducible manner, snapshots cannot be used
>>>>> anymore. In such situations a source dependency come in handy: it
>>>>> is very easy to share and it is as reproducible as a Maven build
>>>>> from a specific commit can be. All CIs and reviewers can work with
>>>>> it, because all source dependency compilation is done under the
>>>>> hood by Maven.
>>>>> 
>>>>> Developers working on changes that span over multiple
>>>>> interdependent git repos can thus get feedback (i-tests, reviews)
>>>>> quickly without waiting for releases of components.
>>>>> 
>>>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>>>> this kind of situation and is in use there since around October
>>>>> 2015.
>>>>> 
>>>>> When I said there is no need to wait for releases of components, I
>>>>> did not mean that we can get rid of component releases altogether.
>>>>> Clearly, we cannot, because i.a. for any tooling uninformed about
>>>>> how srcdeps work, those source dependencies would simply be
>>>>> non-resolvable from public Maven repositories. So, before releasing
>>>>> the dependent component (such as wildfly-core) all its dependencies
>>>>> need to be released. To enforce this, srcdeps is by default
>>>>> configured to make the release fail, as long as there are source
>>>>> dependencies.
>>>>> 
>>>>> I have sent a PR introducing srcdeps to wildfly-core:
>>>>> https://github.com/wildfly/wildfly-core/pull/2122 To get a feeling
>>>>> how it works, checkout the branch, switch to e.g.
>>>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>>>> (that happens to be the commit id of the 1.4.8.Final tag) and
>>>>> build wildfly-core as usual with "mvn clean install". You'll see in
>>>>> the build log that undertow is being cloned to
>>>>> ~/.m2/srcdeps/io/undertow and that it is built there. After the
>>>>> build, check that the
>>>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>>>> version of Undertow got installed to your local Maven repo (usually
>>>>> ~/m2/repository/io/undertow/undertow-core )
>>>>> 
>>>>> Are there any questions or comments?
>>>>> 
>>>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Peter
>>>>> 
>>>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>>>> 14:30 at DevConf Brno.
>>>>> _______________________________________________ wildfly-dev mailing
>>>>> list wildfly-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>> 
>>> 
>> 
> 

-- 
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat





From ppalaga at redhat.com  Mon Jan 30 10:36:18 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Mon, 30 Jan 2017 16:36:18 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <bd68568d-c871-0b40-54fa-0a7d0a1cd44e@jboss.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<cb3c9ece-98db-e1ba-af4f-738968d419af@jboss.com>
	<f0fdcf89-07f7-2c2d-7fbb-a2c1155e0872@redhat.com>
	<bd68568d-c871-0b40-54fa-0a7d0a1cd44e@jboss.com>
Message-ID: <340b7b1e-8fb2-a4da-3d00-68f4e70e4861@redhat.com>

Hi Darran, inline...

On 2017-01-26 13:56, Darran Lofthouse wrote:
>
>
> On 26/01/17 12:42, Peter Palaga wrote:
>> Thanks for the comments, Darran, more inline...
>>
>> On 2017-01-26 12:40, Darran Lofthouse wrote:
>>> On 26/01/17 01:02, Brian Stansberry wrote:
>>>> My only concerns with this would relate to comitting this kind of src
>>>> dependency to the poms in the main branches in the widlfly/wildfly
>>>> and wildfly/wildfly-core repos. We?ve managed to survive up to now
>>>> with little or no need for that kind of thing, so until we get used
>>>> to using this in other ways IMHO we should follow the KISS principle
>>>> and forbid that.
>>>
>>> I like the idea of this being available for teams working on a topic
>>> branch but also think we should avoid this kind of dependency in master
>>> - engineers are continually running builds so I don't think we should
>>> add additional source checkouts and builds.
>>
>> So you also vote for having the support for source dependencies when the
>> e.g. the wildfly-core CI builds a PR that integrates a change in Elytron
>> or when a reviewer does the same, but such PRs should wait for a proper
>> release of the component, right?
>
> I think by the time we get to the PR stage engineers should have had an
> opportunity to discuss the proposed solution sufficiently that we
> shouldn't need a srcdep and should have been able to tag Elytron.

OK, thanks for explaining!

>>> I know Elytron has caused huge amounts of instability but generally I
>>> think we want these branches in a state where we could tag if we
>>> needed to.
>>
>> "these branches" - you mean topic branches in a component (such as
>> Elytron) or in a consuming project (such as WF Core) or both?
>
> I think WildFly Core / master and WildFly / master are generally not
> supposed to be as unstable as we have made them recently.
>
>>> When these downloaded builds are executed are the test cases within also
>>> executed?
>>
>> The builds of source dependencies run with -DskipTests by default so
>> that they finish faster. An explicit per git repository setting in
>> srcdeps.yaml is needed if somebody wants a dependency build to run
>> without -DskipTests.
>>
>>>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>>>> a SRC dependency to get CI or review and accidentally gets merged.
>>>> But I suppose that?s not the end of the world, so long as the release
>>>> process will eventually detect it and fail.
>>>
>>> Merging topic branches these temporary SHAs would be in the history but
>>> maybe the PR CI runs could verify they are not present in the final
>>> merge.
>>>
>>>> Can making srcdeps fail (or just disabling it) be turned on via a
>>>> maven profile? With that we could set up such a profile and turn it
>>>> on in CI jobs that are testing branches where it?s forbidden (e.g.
>>>> the nightly builds of master.)
>>>>
>>>> Oh, one other concern ? how robust is this in the face of poor
>>>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml. If
>>>> that gets out of date or something is the only effect that using a
>>>> src dependency for the affected item doesn't work?
>>>
>>> I think that file would be better almost empty
>>
>> The size of the srcdeps.yaml file depends on how many dependencies the
>> given project has and which of those are selected for the srcdeps
>> support. In the PR https://git.io/vMjUC , I have included only the
>> projects owned by us, excluding non-Maven and non-git projects. I am
>> open to discussion whether we should include or exclude more projects.
>>
>>> but not hidden.
>>
>> The .mvn directory was introduced by Maven 3.3.1. Maven reads
>> extensions.xml file from there and I found it rather natural to have
>> srcdeps.yaml file there too.
>
> What that means is that a file that can directly influence the outcome
> of a local build is hidden from the engineer.  There are already quite a
> few things in out builds where when they go wrong we have to spend time
> tracking them down, having a hidden configuration may not help this.

If I heart this earlier, I'd most probably choose a non-hidden location. 
I perhaps do not see this as a problem, because I configure each of my 
Eclipse workspaces to show all .* resources because .mvn is not the only 
important file that is hidden: there is also .gitignore, .gitattributes, 
.travis.yml, .appveyor.yml and even .git and others that one wants to 
open from time to time.
But now that srcdeps has some users, sorry, I do not think I find this 
important enough to break backwards compatibility.

-- P

>>> Thinking about the topic branch development, within the Elytron team we
>>> are using some SNAPSHOTs from the official repos and at times we have
>>> forked projects into our incubator and developed against SNAPSHOTs from
>>> those
>>
>> Allowing SNAPSHOTs on any remote Maven repository leads to loosing
>> reproducibility of the builds of the dependent project. Srcdeps offers
>> much more reproducibility and portalbility over environments in such
>> situations.
>
> Yes I am saying in the development of the Elytron topic branches this
> could help a lot - but the assumptions as to which git repos we are
> using would be wrong.
>
>>> as as you select the SHA for the build you would also need to
>>> select which repository you really want.
>>
>> Yes, the mapping which dependencies are built from which git
>> repositories is defined in srcdeps.yaml. I see no problem in having one
>> git URL in one branch and having different git URL in another branch.
>
> Which I think as this is a file engineers could be editing to tweak
> their builds means it should not be hidden.
 >
>> Thanks,
>>
>> Peter
>>
>>>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com> wrote:
>>>>>
>>>>> Hi *,
>>>>>
>>>>> this is not new to those of you who attended my talk on the F2F
>>>>> 2016 in
>>>>> Brno. Let me explain the idea here again for all others who did not
>>>>> have
>>>>> a chance to be there.
>>>>>
>>>>> Srcdeps [1] is a tool to build Maven dependencies from their sources.
>>>>> With srcdeps, wildfly-core can depend on a specific commit of, e.g.,
>>>>> undertow:
>>>>>
>>>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>>>
>>>>>
>>>>>
>>>>> where aabbccd is the git commit id to build when any undertow artifact
>>>>> is requested during the build of wildfly-core.
>>>>>
>>>>> [1] describes in detail, how it works.
>>>>>
>>>>> The main advantage of srcdeps is that changes in components can be
>>>>> integrated and tested in wildfly-core immediately after they are
>>>>> committed to a public component branch. There is no need to wait for
>>>>> the
>>>>> component release.
>>>>>
>>>>> Here in the WildFly family of projects, it is often the case that
>>>>> something needs to be fixed in a component, but the verification
>>>>> (using
>>>>> bug reproducer, or integration test) is possible only at the level of
>>>>> wildfly or wildfly-core. Engineers typically work with snapshots
>>>>> locally, but when their changes need to get shared (CI, reviews) in a
>>>>> reproducible manner, snapshots cannot be used anymore.
>>>>> In such situations a source dependency come in handy: it is very
>>>>> easy to
>>>>> share and it is as reproducible as a Maven build from a specific
>>>>> commit
>>>>> can be. All CIs and reviewers can work with it, because all source
>>>>> dependency compilation is done under the hood by Maven.
>>>>>
>>>>> Developers working on changes that span over multiple interdependent
>>>>> git
>>>>> repos can thus get feedback (i-tests, reviews) quickly without waiting
>>>>> for releases of components.
>>>>>
>>>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>>>> this
>>>>> kind of situation and is in use there since around October 2015.
>>>>>
>>>>> When I said there is no need to wait for releases of components, I did
>>>>> not mean that we can get rid of component releases altogether.
>>>>> Clearly,
>>>>> we cannot, because i.a. for any tooling uninformed about how srcdeps
>>>>> work, those source dependencies would simply be non-resolvable from
>>>>> public Maven repositories. So, before releasing the dependent
>>>>> component
>>>>> (such as wildfly-core) all its dependencies need to be released. To
>>>>> enforce this, srcdeps is by default configured to make the release
>>>>> fail,
>>>>> as long as there are source dependencies.
>>>>>
>>>>> I have sent a PR introducing srcdeps to wildfly-core:
>>>>> https://github.com/wildfly/wildfly-core/pull/2122
>>>>> To get a feeling how it works, checkout the branch, switch to e.g.
>>>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>>>>
>>>>>
>>>>> (that happens to be the commit id of the 1.4.8.Final tag)
>>>>> and build wildfly-core as usual with "mvn clean install". You'll
>>>>> see in
>>>>> the build log that undertow is being cloned to
>>>>> ~/.m2/srcdeps/io/undertow
>>>>> and that it is built there. After the build, check that the
>>>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>>>> version of Undertow got installed to your local Maven repo (usually
>>>>> ~/m2/repository/io/undertow/undertow-core )
>>>>>
>>>>> Are there any questions or comments?
>>>>>
>>>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Peter
>>>>>
>>>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>>>> 14:30 at
>>>>> DevConf Brno.
>>>>> _______________________________________________
>>>>> wildfly-dev mailing list
>>>>> wildfly-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>


From ppalaga at redhat.com  Mon Jan 30 10:37:33 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Mon, 30 Jan 2017 16:37:33 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <A567C0B1-5DC3-4ACD-97B6-CCA7F4194F6D@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<d877a377-f183-2aca-40ff-50a12c06c09e@redhat.com>
	<1EC27C97-CC18-4CB6-A434-2E053A8999CE@redhat.com>
	<60823934-405c-c800-074e-86f6da854e32@redhat.com>
	<A567C0B1-5DC3-4ACD-97B6-CCA7F4194F6D@redhat.com>
Message-ID: <6fbf9ccd-9354-86f4-7926-182d692b7d0d@redhat.com>

Thanks everybody for the feedback!

Let me try to sum up the results of the discussion so far:

(1) Nobody except for me wants commits with source dependencies in 
stable branches (such as master or 2.x in wildfly-core). I fully accept 
that the ability to release anytime would go away with source dependency 
merges and I thus give up in this point as long as the "release anytime" 
requirement will be there.

(2) There seems to exist some (enough?) agreement that source 
dependencies could be allowed in pull requests. Such pull requests would 
be there to allow fast feedback from the CI and reviewers, even before 
the release of the dependency. But given (1), such PRs would have to be 
upgraded to a proper dependency release before merging.

(3) If nobody vetoes (2), I am going to implement a failWithout 
configuration option in srcdeps.yaml that will allow for making srcdeps 
resolution opt-in (e.g. via -Dsrcdeps.enabled) for those CI jobs that 
build from PRs but will keep all other srcdeps builds failing.

Having (3) will not prevent merges of PRs with source dependencies 
directly, but will at least make the after-merge CI job fail so that the 
maintainer of the branch is informed very quickly that something bad 
happened.

(4) I'd also add some sort of code (probably a Maven Mojo) to srcdeps 
that would allow to figure out for a given Maven project if it uses 
source dependencies. This could be used to label PRs so that the 
gatekeeper sees clearly that the given PR has source dependencies.

Thanks,

Peter


On 2017-01-26 19:11, Brian Stansberry wrote:
>
>> On Jan 26, 2017, at 10:28 AM, Peter Palaga <ppalaga at redhat.com> wrote:
>>
>> On 2017-01-26 15:44, Brian Stansberry wrote:
>>> There?s been a lot of discussion overnight, but I?ll reply to this one directly since my answers better align with your questions here. :)
>>>
>>>> On Jan 26, 2017, at 2:54 AM, Peter Palaga <ppalaga at redhat.com> wrote:
>>>>
>>>> Hi Brian, thanks for your comments, more inline...
>>>>
>>>> On 2017-01-26 02:02, Brian Stansberry wrote:
>>>>> My only concerns with this would relate to comitting this kind of src
>>>>> dependency to the poms in the main branches in the widlfly/wildfly
>>>>> and wildfly/wildfly-core repos. We?ve managed to survive up to now
>>>>> with little or no need for that kind of thing, so until we get used
>>>>> to using this in other ways IMHO we should follow the KISS principle
>>>>> and forbid that.
>>>>
>>>> Maybe I overestimate the amount of changes that span over multiple git repos. Maybe you in the Core team do not do this often. But for us in the Sustaining Engineering Team, this is quite a typical situation. A substantial part of the reports from customers come with a description how to reproduce on the whole server, but they need to be fixed in a component. Having srcdeps would make the CP process simpler and faster, allowing us to uncover the conflicts and regressions earlier.
>>>
>>> I don?t see how merging to the main branches is required to get this benefit. Git topic branches are fully sharable and CI jobs against them are easily done. All CI tests of pull requests are tests of topic branches.
>>
>> Yes, for me as the submitter of the PR, it is nice to get the feedback from the CI and a review early, even before the component is released, but it is quite bothersome to have to revisit the PR again once the component gets released and rebase (in case there there are conflicts) and either upgrade to the released component version or remove the upgrade change (if the upgrade was merged separately).
>>
>> As long as my PR is not merged, my changes are not binding for the rest of the team. I want my PR to get merged as fast as possible and make others care that their changes are compatible with mine. I want to happily forget about the PR as soon as possible and pick a new task :)
>>
>
> You?ve convinced me! Convinced me that we shouldn?t allow this. :D
>
> We don?t have a role analogous to the ?release coordinator? used with EAP CPs, i.e. someone whose primary responsibility is coordinating to make sure that the untidy pieces get tidied. Most of our non-CR/Final releases are done as side tasks by people who are stealing time from other tasks. They need to be simple and mechanical. We also have a far greater volume of changes to manage than EAP CPs do. A process based on merging half the necessary change and then letting the issue owner walk away and assume someone else is going to come tidy up is a recipe for disaster.
>
>>> But, in any case perhaps you?ve seen clear need for merging to the main branches with the EAP CP branches. I haven?t seen it in WildFly / WildFly Core. I deliberately used specific repo names in my last comment to try and scope it. ;)
>>
>> My reasons for merging there in EAP CP branches are the same as here in the community branches: it is better for PR submitters to merge as early as possible to avoid conflicts, subsequent PR edits and to keep the list of open tasks short.
>>
>>> Note I?m not saying we should disallow PRs with src deps in the pom. We should just disallow merging until those are replaced.
>>
>> Yes, I understand that and I appreciate that. That would be a progress too.
>>
>>>>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>>>>> a SRC dependency to get CI or review and accidentally gets merged.
>>>>
>>>> Oh, I am just realizing I have not said anything about merging. I actually do want to propose that commits with source dependencies get merged to e.g. wildfly-core master as early as possible. Those are the key points of Continuous Integration: get feedback quickly, and merge as soon as possible. This is exactly what Hawkular is doing since more than a year.
>>>
>>> We regularly produce releases (ideally weekly for WildFly Core), often at short notice under pressure.  Allowing merging of changes that are not acceptable for release increases the risk and effort required to do that, since now we have to scan for src deps and figure out how to get them out of the build. Perhaps needing assistance from whoever added the src dep and the lead of relevant component, both of whom are on the other side of the world asleep. (This is a real issue since we often do releases on Friday afternoon US time or Monday morning European time.) We already have too much risk and effort doing releases so adding more will need a really strong justification.
>>
>> This sounds as a valid concern. I must admit I know little about how you plan and perform the releases of wildfly-core, wildfly and of the components in the community. Knowing how complex the graph of WF components is, I am far from underestimating any manual release efforts or efforts to setup a CI jobs to do that automagically. I'll have to gather more info about how you work.
>>
>>>>> But I suppose that?s not the end of the world, so long as the release
>>>>> process will eventually detect it and fail.
>>>>
>>>> Yes, source dependencies on a stable branch do not harm. They just need to be avoided in releases (for which srcdeps offers technical means).
>>>
>>> They do do harm as they mean the branch is no longer releasable. It?s not end-of-the-world harm but it?s harm.
>>
>> Well, I naivelly thought, that the components are obligated to provide a release, say, one day before a planned wildfly-core release and send a PRs that would then sweep out all source dependencies. And TBH, I did not think "releasable at any time" is important in wildfly-core. "Releasable once a week" still sounds good enough to me :)
>
> Unfortunately, it?s not.
>
>>
>>>>> Can making srcdeps fail (or just disabling it) be turned on via a
>>>>> maven profile? With that we could set up such a profile and turn it
>>>>> on in CI jobs that are testing branches where it?s forbidden (e.g.
>>>>> the nightly builds of master.)
>>>>
>>>> Yes, the feature is called "failWith profiles" and can be configured in .mvn/srcdeps.yaml, like here in this srcdeps quickstart: https://github.com/srcdeps/srcdeps-maven/blob/master/srcdeps-maven-quickstarts/srcdeps-mvn-git-profile-quickstart/.mvn/srcdeps.yaml#L33
>>>> There is also "failWith properties" and "failWith goals". It is documented here: https://github.com/srcdeps/srcdeps-core/blob/master/doc/srcdeps.yaml#L130
>>>> By default there is failWith: {goals: release:prepare, release:perform}. Projects that do not use the release plugin can set e.g. failWith: {goals: deploy:deploy} or whatever else distinguishes their releases.
>>>>
>>>
>>> Thanks.
>>>
>>>>> Oh, one other concern ? how robust is this in the face of poor
>>>>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml.
>>>>
>>>> Which parts are boilerpate?
>>>
>>> All of it. :)
>>>
>>> I?m not using that word as an attack. I?m just saying it?s extra text that needs to be maintained, and since it?s separate from the usual place similar text occurs (the poms) it is more likely to diverge.
>>
>> OK, now I know what you mean :) You are right that poms can diverge from srcdeps.yaml.
>>
>>>>> If
>>>>> that gets out of date or something is the only effect that using a
>>>>> src dependency for the affected item doesn't work?
>>>>
>>>> Yes, I think so. As long as the .mvn/srcdeps.yaml file is syntactically correct, any misconfiguration there should not have any other effect than eventually breaking an embedded build.
>>>>
>>>> Generally, the things configured in .mvn/srcdeps.yaml tend to be quite stable - it is basically just mapping from GAVs to their respective git URLs. Git URLs do not change often. It is true that dependency artifacts come and go, but as long as their groupIds are selected reasonably (one groupId occurs in not more than one git repo) the mapping itself can be quite stable over time too.
>>>
>>> Yeah, that?s true. Where this file would be more likely to go unmaintained is adding new entries or cleaning out old ones. But the latter is just noise and if the only harm of the former is a srcdep can?t be used for that lib, then that will naturally get handled by whoever wants to use the srcdep.
>>
>> Yes, exactly.
>>
>> Thanks,
>>
>> Peter
>>
>>>>
>>>> Thanks,
>>>>
>>>> Peter
>>>>
>>>>>
>>>>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hi *,
>>>>>>
>>>>>> this is not new to those of you who attended my talk on the F2F
>>>>>> 2016 in Brno. Let me explain the idea here again for all others who
>>>>>> did not have a chance to be there.
>>>>>>
>>>>>> Srcdeps [1] is a tool to build Maven dependencies from their
>>>>>> sources. With srcdeps, wildfly-core can depend on a specific commit
>>>>>> of, e.g., undertow:
>>>>>>
>>>>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>>>>
>>>>>>
>>>>>>
>>>> where aabbccd is the git commit id to build when any undertow artifact
>>>>>> is requested during the build of wildfly-core.
>>>>>>
>>>>>> [1] describes in detail, how it works.
>>>>>>
>>>>>> The main advantage of srcdeps is that changes in components can be
>>>>>> integrated and tested in wildfly-core immediately after they are
>>>>>> committed to a public component branch. There is no need to wait
>>>>>> for the component release.
>>>>>>
>>>>>> Here in the WildFly family of projects, it is often the case that
>>>>>> something needs to be fixed in a component, but the verification
>>>>>> (using bug reproducer, or integration test) is possible only at the
>>>>>> level of wildfly or wildfly-core. Engineers typically work with
>>>>>> snapshots locally, but when their changes need to get shared (CI,
>>>>>> reviews) in a reproducible manner, snapshots cannot be used
>>>>>> anymore. In such situations a source dependency come in handy: it
>>>>>> is very easy to share and it is as reproducible as a Maven build
>>>>>> from a specific commit can be. All CIs and reviewers can work with
>>>>>> it, because all source dependency compilation is done under the
>>>>>> hood by Maven.
>>>>>>
>>>>>> Developers working on changes that span over multiple
>>>>>> interdependent git repos can thus get feedback (i-tests, reviews)
>>>>>> quickly without waiting for releases of components.
>>>>>>
>>>>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>>>>> this kind of situation and is in use there since around October
>>>>>> 2015.
>>>>>>
>>>>>> When I said there is no need to wait for releases of components, I
>>>>>> did not mean that we can get rid of component releases altogether.
>>>>>> Clearly, we cannot, because i.a. for any tooling uninformed about
>>>>>> how srcdeps work, those source dependencies would simply be
>>>>>> non-resolvable from public Maven repositories. So, before releasing
>>>>>> the dependent component (such as wildfly-core) all its dependencies
>>>>>> need to be released. To enforce this, srcdeps is by default
>>>>>> configured to make the release fail, as long as there are source
>>>>>> dependencies.
>>>>>>
>>>>>> I have sent a PR introducing srcdeps to wildfly-core:
>>>>>> https://github.com/wildfly/wildfly-core/pull/2122 To get a feeling
>>>>>> how it works, checkout the branch, switch to e.g.
>>>>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>>>>> (that happens to be the commit id of the 1.4.8.Final tag) and
>>>>>> build wildfly-core as usual with "mvn clean install". You'll see in
>>>>>> the build log that undertow is being cloned to
>>>>>> ~/.m2/srcdeps/io/undertow and that it is built there. After the
>>>>>> build, check that the
>>>>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>>>>> version of Undertow got installed to your local Maven repo (usually
>>>>>> ~/m2/repository/io/undertow/undertow-core )
>>>>>>
>>>>>> Are there any questions or comments?
>>>>>>
>>>>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Peter
>>>>>>
>>>>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>>>>> 14:30 at DevConf Brno.
>>>>>> _______________________________________________ wildfly-dev mailing
>>>>>> list wildfly-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>>
>>>>
>>>
>>
>


From darran.lofthouse at jboss.com  Mon Jan 30 11:29:28 2017
From: darran.lofthouse at jboss.com (Darran Lofthouse)
Date: Mon, 30 Jan 2017 16:29:28 +0000
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <340b7b1e-8fb2-a4da-3d00-68f4e70e4861@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<cb3c9ece-98db-e1ba-af4f-738968d419af@jboss.com>
	<f0fdcf89-07f7-2c2d-7fbb-a2c1155e0872@redhat.com>
	<bd68568d-c871-0b40-54fa-0a7d0a1cd44e@jboss.com>
	<340b7b1e-8fb2-a4da-3d00-68f4e70e4861@redhat.com>
Message-ID: <cc948d63-d608-a4f7-a705-d99caccf06bc@jboss.com>

On 30/01/17 15:36, Peter Palaga wrote:
> Hi Darran, inline...
>
> On 2017-01-26 13:56, Darran Lofthouse wrote:
>> What that means is that a file that can directly influence the outcome
>> of a local build is hidden from the engineer.  There are already quite a
>> few things in out builds where when they go wrong we have to spend time
>> tracking them down, having a hidden configuration may not help this.
>
> If I heart this earlier, I'd most probably choose a non-hidden location.
> I perhaps do not see this as a problem, because I configure each of my
> Eclipse workspaces to show all .* resources because .mvn is not the only
> important file that is hidden: there is also .gitignore, .gitattributes,
> .travis.yml, .appveyor.yml and even .git and others that one wants to
> open from time to time.
> But now that srcdeps has some users, sorry, I do not think I find this
> important enough to break backwards compatibility.

But those files listed don't influence the local build.

Regarding backwards compatibility, what backwards compatibility are you 
referring to as from what I understand the PRs into WildFly and WildFly 
Core have not been merged yet.

>
> -- P
>

From ppalaga at redhat.com  Mon Jan 30 14:11:37 2017
From: ppalaga at redhat.com (Peter Palaga)
Date: Mon, 30 Jan 2017 20:11:37 +0100
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <cc948d63-d608-a4f7-a705-d99caccf06bc@jboss.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<cb3c9ece-98db-e1ba-af4f-738968d419af@jboss.com>
	<f0fdcf89-07f7-2c2d-7fbb-a2c1155e0872@redhat.com>
	<bd68568d-c871-0b40-54fa-0a7d0a1cd44e@jboss.com>
	<340b7b1e-8fb2-a4da-3d00-68f4e70e4861@redhat.com>
	<cc948d63-d608-a4f7-a705-d99caccf06bc@jboss.com>
Message-ID: <c525b99d-0348-107b-9b56-7ddcdd75856a@redhat.com>

On 2017-01-30 17:29, Darran Lofthouse wrote:
> On 30/01/17 15:36, Peter Palaga wrote:
>> Hi Darran, inline...
>>
>> On 2017-01-26 13:56, Darran Lofthouse wrote:
>>> What that means is that a file that can directly influence the outcome
>>> of a local build is hidden from the engineer.  There are already quite a
>>> few things in out builds where when they go wrong we have to spend time
>>> tracking them down, having a hidden configuration may not help this.
>>
>> If I heart this earlier, I'd most probably choose a non-hidden location.
>> I perhaps do not see this as a problem, because I configure each of my
>> Eclipse workspaces to show all .* resources because .mvn is not the only
>> important file that is hidden: there is also .gitignore, .gitattributes,
>> .travis.yml, .appveyor.yml and even .git and others that one wants to
>> open from time to time.
>> But now that srcdeps has some users, sorry, I do not think I find this
>> important enough to break backwards compatibility.
>
> But those files listed don't influence the local build.
>
> Regarding backwards compatibility, what backwards compatibility are you
> referring to as from what I understand the PRs into WildFly and WildFly
> Core have not been merged yet.

srcdeps is used by other non-WF projects already and changing the 
location where it looks for its configuration file would either break 
those projects or require a migration. -- P


From darran.lofthouse at jboss.com  Mon Jan 30 14:29:13 2017
From: darran.lofthouse at jboss.com (Darran Lofthouse)
Date: Mon, 30 Jan 2017 19:29:13 +0000
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <c525b99d-0348-107b-9b56-7ddcdd75856a@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<cb3c9ece-98db-e1ba-af4f-738968d419af@jboss.com>
	<f0fdcf89-07f7-2c2d-7fbb-a2c1155e0872@redhat.com>
	<bd68568d-c871-0b40-54fa-0a7d0a1cd44e@jboss.com>
	<340b7b1e-8fb2-a4da-3d00-68f4e70e4861@redhat.com>
	<cc948d63-d608-a4f7-a705-d99caccf06bc@jboss.com>
	<c525b99d-0348-107b-9b56-7ddcdd75856a@redhat.com>
Message-ID: <02a1df70-a0f1-b5ec-4684-6dd777afb47f@jboss.com>



On 30/01/17 19:11, Peter Palaga wrote:
> On 2017-01-30 17:29, Darran Lofthouse wrote:
>> On 30/01/17 15:36, Peter Palaga wrote:
>>> Hi Darran, inline...
>>>
>>> On 2017-01-26 13:56, Darran Lofthouse wrote:
>>>> What that means is that a file that can directly influence the outcome
>>>> of a local build is hidden from the engineer.  There are already
>>>> quite a
>>>> few things in out builds where when they go wrong we have to spend time
>>>> tracking them down, having a hidden configuration may not help this.
>>>
>>> If I heart this earlier, I'd most probably choose a non-hidden location.
>>> I perhaps do not see this as a problem, because I configure each of my
>>> Eclipse workspaces to show all .* resources because .mvn is not the only
>>> important file that is hidden: there is also .gitignore, .gitattributes,
>>> .travis.yml, .appveyor.yml and even .git and others that one wants to
>>> open from time to time.
>>> But now that srcdeps has some users, sorry, I do not think I find this
>>> important enough to break backwards compatibility.
>>
>> But those files listed don't influence the local build.
>>
>> Regarding backwards compatibility, what backwards compatibility are you
>> referring to as from what I understand the PRs into WildFly and WildFly
>> Core have not been merged yet.
>
> srcdeps is used by other non-WF projects already and changing the
> location where it looks for its configuration file would either break
> those projects or require a migration. -- P

There are ways to solve that problem, but if no one else feels it should 
not be hidden I understand.

>

From kabir.khan at jboss.com  Tue Jan 31 06:03:50 2017
From: kabir.khan at jboss.com (Kabir Khan)
Date: Tue, 31 Jan 2017 11:03:50 +0000
Subject: [wildfly-dev] Srcdeps in WildFly and WildFly Core
In-Reply-To: <340b7b1e-8fb2-a4da-3d00-68f4e70e4861@redhat.com>
References: <ba9c63b8-53ee-7354-9f87-eb1598024986@redhat.com>
	<BE4FF1E1-B553-418B-8563-FBFC8230DEB7@redhat.com>
	<cb3c9ece-98db-e1ba-af4f-738968d419af@jboss.com>
	<f0fdcf89-07f7-2c2d-7fbb-a2c1155e0872@redhat.com>
	<bd68568d-c871-0b40-54fa-0a7d0a1cd44e@jboss.com>
	<340b7b1e-8fb2-a4da-3d00-68f4e70e4861@redhat.com>
Message-ID: <6E9564A3-3A0F-4F9E-9034-D88580563623@jboss.com>


> On 30 Jan 2017, at 15:36, Peter Palaga <ppalaga at redhat.com> wrote:
> 
> Hi Darran, inline...
> 
> On 2017-01-26 13:56, Darran Lofthouse wrote:
>> 
>> 
>> On 26/01/17 12:42, Peter Palaga wrote:
>>> Thanks for the comments, Darran, more inline...
>>> 
>>> On 2017-01-26 12:40, Darran Lofthouse wrote:
>>>> On 26/01/17 01:02, Brian Stansberry wrote:
>>>>> My only concerns with this would relate to comitting this kind of src
>>>>> dependency to the poms in the main branches in the widlfly/wildfly
>>>>> and wildfly/wildfly-core repos. We?ve managed to survive up to now
>>>>> with little or no need for that kind of thing, so until we get used
>>>>> to using this in other ways IMHO we should follow the KISS principle
>>>>> and forbid that.
>>>> 
>>>> I like the idea of this being available for teams working on a topic
>>>> branch but also think we should avoid this kind of dependency in master
>>>> - engineers are continually running builds so I don't think we should
>>>> add additional source checkouts and builds.
>>> 
>>> So you also vote for having the support for source dependencies when the
>>> e.g. the wildfly-core CI builds a PR that integrates a change in Elytron
>>> or when a reviewer does the same, but such PRs should wait for a proper
>>> release of the component, right?
>> 
>> I think by the time we get to the PR stage engineers should have had an
>> opportunity to discuss the proposed solution sufficiently that we
>> shouldn't need a srcdep and should have been able to tag Elytron.
> 
> OK, thanks for explaining!
> 
>>>> I know Elytron has caused huge amounts of instability but generally I
>>>> think we want these branches in a state where we could tag if we
>>>> needed to.
>>> 
>>> "these branches" - you mean topic branches in a component (such as
>>> Elytron) or in a consuming project (such as WF Core) or both?
>> 
>> I think WildFly Core / master and WildFly / master are generally not
>> supposed to be as unstable as we have made them recently.
>> 
>>>> When these downloaded builds are executed are the test cases within also
>>>> executed?
>>> 
>>> The builds of source dependencies run with -DskipTests by default so
>>> that they finish faster. An explicit per git repository setting in
>>> srcdeps.yaml is needed if somebody wants a dependency build to run
>>> without -DskipTests.
>>> 
>>>>> A trick is avoiding doing that by mistake; i.e. a PR is sent up with
>>>>> a SRC dependency to get CI or review and accidentally gets merged.
>>>>> But I suppose that?s not the end of the world, so long as the release
>>>>> process will eventually detect it and fail.
>>>> 
>>>> Merging topic branches these temporary SHAs would be in the history but
>>>> maybe the PR CI runs could verify they are not present in the final
>>>> merge.
>>>> 
>>>>> Can making srcdeps fail (or just disabling it) be turned on via a
>>>>> maven profile? With that we could set up such a profile and turn it
>>>>> on in CI jobs that are testing branches where it?s forbidden (e.g.
>>>>> the nightly builds of master.)
>>>>> 
>>>>> Oh, one other concern ? how robust is this in the face of poor
>>>>> maintenance? I see a lot of boilerplate in that .mvn/srcdeps.yaml. If
>>>>> that gets out of date or something is the only effect that using a
>>>>> src dependency for the affected item doesn't work?
>>>> 
>>>> I think that file would be better almost empty
>>> 
>>> The size of the srcdeps.yaml file depends on how many dependencies the
>>> given project has and which of those are selected for the srcdeps
>>> support. In the PR https://git.io/vMjUC , I have included only the
>>> projects owned by us, excluding non-Maven and non-git projects. I am
>>> open to discussion whether we should include or exclude more projects.
>>> 
>>>> but not hidden.
>>> 
>>> The .mvn directory was introduced by Maven 3.3.1. Maven reads
>>> extensions.xml file from there and I found it rather natural to have
>>> srcdeps.yaml file there too.
>> 
>> What that means is that a file that can directly influence the outcome
>> of a local build is hidden from the engineer.  There are already quite a
>> few things in out builds where when they go wrong we have to spend time
>> tracking them down, having a hidden configuration may not help this.
> 
> If I heart this earlier, I'd most probably choose a non-hidden location. 
> I perhaps do not see this as a problem, because I configure each of my 
> Eclipse workspaces to show all .* resources because .mvn is not the only 
> important file that is hidden: there is also .gitignore, .gitattributes, 
> .travis.yml, .appveyor.yml and even .git and others that one wants to 
> open from time to time.
> But now that srcdeps has some users, sorry, I do not think I find this 
> important enough to break backwards compatibility.
You could perhaps provide both? It will first look in the hidden location, and next in the visible location (wherever that may be). If both are used, give an error.
> 
> -- P
> 
>>>> Thinking about the topic branch development, within the Elytron team we
>>>> are using some SNAPSHOTs from the official repos and at times we have
>>>> forked projects into our incubator and developed against SNAPSHOTs from
>>>> those
>>> 
>>> Allowing SNAPSHOTs on any remote Maven repository leads to loosing
>>> reproducibility of the builds of the dependent project. Srcdeps offers
>>> much more reproducibility and portalbility over environments in such
>>> situations.
>> 
>> Yes I am saying in the development of the Elytron topic branches this
>> could help a lot - but the assumptions as to which git repos we are
>> using would be wrong.
>> 
>>>> as as you select the SHA for the build you would also need to
>>>> select which repository you really want.
>>> 
>>> Yes, the mapping which dependencies are built from which git
>>> repositories is defined in srcdeps.yaml. I see no problem in having one
>>> git URL in one branch and having different git URL in another branch.
>> 
>> Which I think as this is a file engineers could be editing to tweak
>> their builds means it should not be hidden.
>> 
>>> Thanks,
>>> 
>>> Peter
>>> 
>>>>>> On Jan 25, 2017, at 3:45 PM, Peter Palaga <ppalaga at redhat.com> wrote:
>>>>>> 
>>>>>> Hi *,
>>>>>> 
>>>>>> this is not new to those of you who attended my talk on the F2F
>>>>>> 2016 in
>>>>>> Brno. Let me explain the idea here again for all others who did not
>>>>>> have
>>>>>> a chance to be there.
>>>>>> 
>>>>>> Srcdeps [1] is a tool to build Maven dependencies from their sources.
>>>>>> With srcdeps, wildfly-core can depend on a specific commit of, e.g.,
>>>>>> undertow:
>>>>>> 
>>>>>> <version.io.undertow>1.4.8.Final-SRC-revision-aabbccd</version.io.undertow>
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> where aabbccd is the git commit id to build when any undertow artifact
>>>>>> is requested during the build of wildfly-core.
>>>>>> 
>>>>>> [1] describes in detail, how it works.
>>>>>> 
>>>>>> The main advantage of srcdeps is that changes in components can be
>>>>>> integrated and tested in wildfly-core immediately after they are
>>>>>> committed to a public component branch. There is no need to wait for
>>>>>> the
>>>>>> component release.
>>>>>> 
>>>>>> Here in the WildFly family of projects, it is often the case that
>>>>>> something needs to be fixed in a component, but the verification
>>>>>> (using
>>>>>> bug reproducer, or integration test) is possible only at the level of
>>>>>> wildfly or wildfly-core. Engineers typically work with snapshots
>>>>>> locally, but when their changes need to get shared (CI, reviews) in a
>>>>>> reproducible manner, snapshots cannot be used anymore.
>>>>>> In such situations a source dependency come in handy: it is very
>>>>>> easy to
>>>>>> share and it is as reproducible as a Maven build from a specific
>>>>>> commit
>>>>>> can be. All CIs and reviewers can work with it, because all source
>>>>>> dependency compilation is done under the hood by Maven.
>>>>>> 
>>>>>> Developers working on changes that span over multiple interdependent
>>>>>> git
>>>>>> repos can thus get feedback (i-tests, reviews) quickly without waiting
>>>>>> for releases of components.
>>>>>> 
>>>>>> Srcdeps emerged in the Hawkular family of projects to solve exactly
>>>>>> this
>>>>>> kind of situation and is in use there since around October 2015.
>>>>>> 
>>>>>> When I said there is no need to wait for releases of components, I did
>>>>>> not mean that we can get rid of component releases altogether.
>>>>>> Clearly,
>>>>>> we cannot, because i.a. for any tooling uninformed about how srcdeps
>>>>>> work, those source dependencies would simply be non-resolvable from
>>>>>> public Maven repositories. So, before releasing the dependent
>>>>>> component
>>>>>> (such as wildfly-core) all its dependencies need to be released. To
>>>>>> enforce this, srcdeps is by default configured to make the release
>>>>>> fail,
>>>>>> as long as there are source dependencies.
>>>>>> 
>>>>>> I have sent a PR introducing srcdeps to wildfly-core:
>>>>>> https://github.com/wildfly/wildfly-core/pull/2122
>>>>>> To get a feeling how it works, checkout the branch, switch to e.g.
>>>>>> <version.io.undertow>1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd</version.io.undertow>
>>>>>> 
>>>>>> 
>>>>>> (that happens to be the commit id of the 1.4.8.Final tag)
>>>>>> and build wildfly-core as usual with "mvn clean install". You'll
>>>>>> see in
>>>>>> the build log that undertow is being cloned to
>>>>>> ~/.m2/srcdeps/io/undertow
>>>>>> and that it is built there. After the build, check that the
>>>>>> 1.4.8.Final-SRC-revision-1bff8c32f0eee986e83a7589ae95ebbc1d67d6bd
>>>>>> version of Undertow got installed to your local Maven repo (usually
>>>>>> ~/m2/repository/io/undertow/undertow-core )
>>>>>> 
>>>>>> Are there any questions or comments?
>>>>>> 
>>>>>> [1] https://github.com/srcdeps/srcdeps-maven#srcdeps-maven
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> Peter
>>>>>> 
>>>>>> P.S.: I will be talking about srcdeps on Saturday 2017-01-28 at
>>>>>> 14:30 at
>>>>>> DevConf Brno.
>>>>>> _______________________________________________
>>>>>> wildfly-dev mailing list
>>>>>> wildfly-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>> 
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> wildfly-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>> 
>>> 
> 
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev



From kabir.khan at jboss.com  Tue Jan 31 10:00:18 2017
From: kabir.khan at jboss.com (Kabir Khan)
Date: Tue, 31 Jan 2017 15:00:18 +0000
Subject: [wildfly-dev] New invocation merge
Message-ID: <E265BEB5-88E0-42CF-A26B-C3ADEAA0747F@jboss.com>

Hi,

The new invocation library, which is the basis of for the ejb client etc. is currently developed in a branch. There are still about 130-140 test failures, but the team feels it is time to merge to wildfly master at some stage later this week. This will get more visibility of the failures and also lower the barrier of entry for whoever can jump in and help fix the failures.

Are there any objections to merging this? 

>From my point of view we would need to @Ignore the failing tests since there are enough transient failures in our testsuite to make it hard to find usual suspects if the number of failed tests is large. We could set up another CI job against a branch which is master with the @Ignores reverted, and I could keep that up to date as I merge to master, so that we get a good picture of the current testsuite failures.

Thanks,

Kabir

From jason.greene at redhat.com  Tue Jan 31 10:05:55 2017
From: jason.greene at redhat.com (Jason T. Greene)
Date: Tue, 31 Jan 2017 10:05:55 -0500 (EST)
Subject: [wildfly-dev] New invocation merge
In-Reply-To: <E265BEB5-88E0-42CF-A26B-C3ADEAA0747F@jboss.com>
References: <E265BEB5-88E0-42CF-A26B-C3ADEAA0747F@jboss.com>
Message-ID: <4C4AB07B-56B0-4814-A1D4-6F0C0D59D974@redhat.com>

I agree with this approach but we need a way to group these failures in such a way that we know whether or not a change has regressions. 

> On Jan 31, 2017, at 9:03 AM, Kabir Khan <kabir.khan at jboss.com> wrote:
> 
> Hi,
> 
> The new invocation library, which is the basis of for the ejb client etc. is currently developed in a branch. There are still about 130-140 test failures, but the team feels it is time to merge to wildfly master at some stage later this week. This will get more visibility of the failures and also lower the barrier of entry for whoever can jump in and help fix the failures.
> 
> Are there any objections to merging this? 
> 
>> From my point of view we would need to @Ignore the failing tests since there are enough transient failures in our testsuite to make it hard to find usual suspects if the number of failed tests is large. We could set up another CI job against a branch which is master with the @Ignores reverted, and I could keep that up to date as I merge to master, so that we get a good picture of the current testsuite failures.
> 
> Thanks,
> 
> Kabir
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


From jason.greene at redhat.com  Tue Jan 31 10:09:17 2017
From: jason.greene at redhat.com (Jason T. Greene)
Date: Tue, 31 Jan 2017 10:09:17 -0500 (EST)
Subject: [wildfly-dev] New invocation merge
In-Reply-To: <4C4AB07B-56B0-4814-A1D4-6F0C0D59D974@redhat.com>
References: <E265BEB5-88E0-42CF-A26B-C3ADEAA0747F@jboss.com>
	<4C4AB07B-56B0-4814-A1D4-6F0C0D59D974@redhat.com>
Message-ID: <25BED3E9-ADD6-4ED1-966A-CEE4EE32265F@redhat.com>

How about the tests are changed to require a sys prop. That way you don't need a branch.

> On Jan 31, 2017, at 9:07 AM, Jason T. Greene <jason.greene at redhat.com> wrote:
> 
> I agree with this approach but we need a way to group these failures in such a way that we know whether or not a change has regressions. 
> 
>> On Jan 31, 2017, at 9:03 AM, Kabir Khan <kabir.khan at jboss.com> wrote:
>> 
>> Hi,
>> 
>> The new invocation library, which is the basis of for the ejb client etc. is currently developed in a branch. There are still about 130-140 test failures, but the team feels it is time to merge to wildfly master at some stage later this week. This will get more visibility of the failures and also lower the barrier of entry for whoever can jump in and help fix the failures.
>> 
>> Are there any objections to merging this? 
>> 
>>> From my point of view we would need to @Ignore the failing tests since there are enough transient failures in our testsuite to make it hard to find usual suspects if the number of failed tests is large. We could set up another CI job against a branch which is master with the @Ignores reverted, and I could keep that up to date as I merge to master, so that we get a good picture of the current testsuite failures.
>> 
>> Thanks,
>> 
>> Kabir
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


From kabir.khan at jboss.com  Tue Jan 31 10:14:19 2017
From: kabir.khan at jboss.com (Kabir Khan)
Date: Tue, 31 Jan 2017 15:14:19 +0000
Subject: [wildfly-dev] New invocation merge
In-Reply-To: <25BED3E9-ADD6-4ED1-966A-CEE4EE32265F@redhat.com>
References: <E265BEB5-88E0-42CF-A26B-C3ADEAA0747F@jboss.com>
	<4C4AB07B-56B0-4814-A1D4-6F0C0D59D974@redhat.com>
	<25BED3E9-ADD6-4ED1-966A-CEE4EE32265F@redhat.com>
Message-ID: <0D27DD3F-E93F-4EF0-BECB-D430531910DC@jboss.com>

> 
> On 31 Jan 2017, at 15:09, Jason T. Greene <jason.greene at redhat.com> wrote:
> 
> How about the tests are changed to require a sys prop. That way you don't need a branch.

So rather than @Ignore we add something like 

@BeforeClass
public static checkRunInvocationTests() {
	//WFLY-XXXX
	Assume.assumeTrue(System.getProperty("wildfly.temp.disabled-invocation"))
}

> 
>> On Jan 31, 2017, at 9:07 AM, Jason T. Greene <jason.greene at redhat.com> wrote:
>> 
>> I agree with this approach but we need a way to group these failures in such a way that we know whether or not a change has regressions. 
Can you elaborate a little bit?
>> 
>>> On Jan 31, 2017, at 9:03 AM, Kabir Khan <kabir.khan at jboss.com> wrote:
>>> 
>>> Hi,
>>> 
>>> The new invocation library, which is the basis of for the ejb client etc. is currently developed in a branch. There are still about 130-140 test failures, but the team feels it is time to merge to wildfly master at some stage later this week. This will get more visibility of the failures and also lower the barrier of entry for whoever can jump in and help fix the failures.
>>> 
>>> Are there any objections to merging this? 
>>> 
>>>> From my point of view we would need to @Ignore the failing tests since there are enough transient failures in our testsuite to make it hard to find usual suspects if the number of failed tests is large. We could set up another CI job against a branch which is master with the @Ignores reverted, and I could keep that up to date as I merge to master, so that we get a good picture of the current testsuite failures.
>>> 
>>> Thanks,
>>> 
>>> Kabir
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>> 
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev



From jason.greene at redhat.com  Tue Jan 31 10:14:21 2017
From: jason.greene at redhat.com (Jason T. Greene)
Date: Tue, 31 Jan 2017 10:14:21 -0500 (EST)
Subject: [wildfly-dev] New invocation merge
In-Reply-To: <25BED3E9-ADD6-4ED1-966A-CEE4EE32265F@redhat.com>
References: <E265BEB5-88E0-42CF-A26B-C3ADEAA0747F@jboss.com>
	<4C4AB07B-56B0-4814-A1D4-6F0C0D59D974@redhat.com>
	<25BED3E9-ADD6-4ED1-966A-CEE4EE32265F@redhat.com>
Message-ID: <282A6847-A741-44C6-860E-A3B9F995AFC1@redhat.com>

e.g assumeTrue(toBool(System.getProperty(Xxx))) 


> On Jan 31, 2017, at 9:09 AM, Jason T. Greene <jason.greene at redhat.com> wrote:
> 
> How about the tests are changed to require a sys prop. That way you don't need a branch.
> 
>> On Jan 31, 2017, at 9:07 AM, Jason T. Greene <jason.greene at redhat.com> wrote:
>> 
>> I agree with this approach but we need a way to group these failures in such a way that we know whether or not a change has regressions. 
>> 
>>> On Jan 31, 2017, at 9:03 AM, Kabir Khan <kabir.khan at jboss.com> wrote:
>>> 
>>> Hi,
>>> 
>>> The new invocation library, which is the basis of for the ejb client etc. is currently developed in a branch. There are still about 130-140 test failures, but the team feels it is time to merge to wildfly master at some stage later this week. This will get more visibility of the failures and also lower the barrier of entry for whoever can jump in and help fix the failures.
>>> 
>>> Are there any objections to merging this? 
>>> 
>>>> From my point of view we would need to @Ignore the failing tests since there are enough transient failures in our testsuite to make it hard to find usual suspects if the number of failed tests is large. We could set up another CI job against a branch which is master with the @Ignores reverted, and I could keep that up to date as I merge to master, so that we get a good picture of the current testsuite failures.
>>> 
>>> Thanks,
>>> 
>>> Kabir
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>> 
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev


From jason.greene at redhat.com  Tue Jan 31 10:16:45 2017
From: jason.greene at redhat.com (Jason T. Greene)
Date: Tue, 31 Jan 2017 10:16:45 -0500 (EST)
Subject: [wildfly-dev] New invocation merge
In-Reply-To: <0D27DD3F-E93F-4EF0-BECB-D430531910DC@jboss.com>
References: <E265BEB5-88E0-42CF-A26B-C3ADEAA0747F@jboss.com>
	<4C4AB07B-56B0-4814-A1D4-6F0C0D59D974@redhat.com>
	<25BED3E9-ADD6-4ED1-966A-CEE4EE32265F@redhat.com>
	<0D27DD3F-E93F-4EF0-BECB-D430531910DC@jboss.com>
Message-ID: <823EA79D-BC42-442D-BC06-882D1E874E99@redhat.com>

Right exactly. 

What does everyone think about a daily or weekly email of the test output to this list?

On Jan 31, 2017, at 9:14 AM, Kabir Khan <kabir.khan at jboss.com> wrote:

>> 
>> On 31 Jan 2017, at 15:09, Jason T. Greene <jason.greene at redhat.com> wrote:
>> 
>> How about the tests are changed to require a sys prop. That way you don't need a branch.
> 
> So rather than @Ignore we add something like 
> 
> @BeforeClass
> public static checkRunInvocationTests() {
>    //WFLY-XXXX
>    Assume.assumeTrue(System.getProperty("wildfly.temp.disabled-invocation"))
> }
> 
>> 
>>> On Jan 31, 2017, at 9:07 AM, Jason T. Greene <jason.greene at redhat.com> wrote:
>>> 
>>> I agree with this approach but we need a way to group these failures in such a way that we know whether or not a change has regressions. 
> Can you elaborate a little bit?
>>> 
>>>> On Jan 31, 2017, at 9:03 AM, Kabir Khan <kabir.khan at jboss.com> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> The new invocation library, which is the basis of for the ejb client etc. is currently developed in a branch. There are still about 130-140 test failures, but the team feels it is time to merge to wildfly master at some stage later this week. This will get more visibility of the failures and also lower the barrier of entry for whoever can jump in and help fix the failures.
>>>> 
>>>> Are there any objections to merging this? 
>>>> 
>>>>> From my point of view we would need to @Ignore the failing tests since there are enough transient failures in our testsuite to make it hard to find usual suspects if the number of failed tests is large. We could set up another CI job against a branch which is master with the @Ignores reverted, and I could keep that up to date as I merge to master, so that we get a good picture of the current testsuite failures.
>>>> 
>>>> Thanks,
>>>> 
>>>> Kabir
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> wildfly-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>> 
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> wildfly-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> 


From kabir.khan at jboss.com  Tue Jan 31 10:20:15 2017
From: kabir.khan at jboss.com (Kabir Khan)
Date: Tue, 31 Jan 2017 15:20:15 +0000
Subject: [wildfly-dev] New invocation merge
In-Reply-To: <823EA79D-BC42-442D-BC06-882D1E874E99@redhat.com>
References: <E265BEB5-88E0-42CF-A26B-C3ADEAA0747F@jboss.com>
	<4C4AB07B-56B0-4814-A1D4-6F0C0D59D974@redhat.com>
	<25BED3E9-ADD6-4ED1-966A-CEE4EE32265F@redhat.com>
	<0D27DD3F-E93F-4EF0-BECB-D430531910DC@jboss.com>
	<823EA79D-BC42-442D-BC06-882D1E874E99@redhat.com>
Message-ID: <DA4E0E67-A5C7-4B85-A79A-AC70C9E76FBB@jboss.com>


> On 31 Jan 2017, at 15:16, Jason T. Greene <jason.greene at redhat.com> wrote:
> 
> Right exactly. 
> 
> What does everyone think about a daily or weekly email of the test output to this list?
Daily would narrow down the range of suspicious commits. Although that information should still be available on TeamCity itself, I think this makes it more visible.

(I take it this is the elaboration about the grouping of the failures? :))

> 
> On Jan 31, 2017, at 9:14 AM, Kabir Khan <kabir.khan at jboss.com> wrote:
> 
>>> 
>>> On 31 Jan 2017, at 15:09, Jason T. Greene <jason.greene at redhat.com> wrote:
>>> 
>>> How about the tests are changed to require a sys prop. That way you don't need a branch.
>> 
>> So rather than @Ignore we add something like 
>> 
>> @BeforeClass
>> public static checkRunInvocationTests() {
>>   //WFLY-XXXX
>>   Assume.assumeTrue(System.getProperty("wildfly.temp.disabled-invocation"))
>> }
>> 
>>> 
>>>> On Jan 31, 2017, at 9:07 AM, Jason T. Greene <jason.greene at redhat.com> wrote:
>>>> 
>>>> I agree with this approach but we need a way to group these failures in such a way that we know whether or not a change has regressions. 
>> Can you elaborate a little bit?
>>>> 
>>>>> On Jan 31, 2017, at 9:03 AM, Kabir Khan <kabir.khan at jboss.com> wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> The new invocation library, which is the basis of for the ejb client etc. is currently developed in a branch. There are still about 130-140 test failures, but the team feels it is time to merge to wildfly master at some stage later this week. This will get more visibility of the failures and also lower the barrier of entry for whoever can jump in and help fix the failures.
>>>>> 
>>>>> Are there any objections to merging this? 
>>>>> 
>>>>>> From my point of view we would need to @Ignore the failing tests since there are enough transient failures in our testsuite to make it hard to find usual suspects if the number of failed tests is large. We could set up another CI job against a branch which is master with the @Ignores reverted, and I could keep that up to date as I merge to master, so that we get a good picture of the current testsuite failures.
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Kabir
>>>>> _______________________________________________
>>>>> wildfly-dev mailing list
>>>>> wildfly-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>> 
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> wildfly-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>> 



From jason.greene at redhat.com  Tue Jan 31 11:28:15 2017
From: jason.greene at redhat.com (Jason Greene)
Date: Tue, 31 Jan 2017 10:28:15 -0600
Subject: [wildfly-dev] New invocation merge
In-Reply-To: <DA4E0E67-A5C7-4B85-A79A-AC70C9E76FBB@jboss.com>
References: <E265BEB5-88E0-42CF-A26B-C3ADEAA0747F@jboss.com>
	<4C4AB07B-56B0-4814-A1D4-6F0C0D59D974@redhat.com>
	<25BED3E9-ADD6-4ED1-966A-CEE4EE32265F@redhat.com>
	<0D27DD3F-E93F-4EF0-BECB-D430531910DC@jboss.com>
	<823EA79D-BC42-442D-BC06-882D1E874E99@redhat.com>
	<DA4E0E67-A5C7-4B85-A79A-AC70C9E76FBB@jboss.com>
Message-ID: <B9EB632E-07C7-4025-92AF-497D64DFB524@redhat.com>


> On Jan 31, 2017, at 9:20 AM, Kabir Khan <kabir.khan at jboss.com> wrote:
> 
> 
>> On 31 Jan 2017, at 15:16, Jason T. Greene <jason.greene at redhat.com> wrote:
>> 
>> Right exactly. 
>> 
>> What does everyone think about a daily or weekly email of the test output to this list?
> Daily would narrow down the range of suspicious commits. Although that information should still be available on TeamCity itself, I think this makes it more visible.
> 
> (I take it this is the elaboration about the grouping of the failures? :))

Right I just mean if the goal is to recruit help in resolving test failures, then highlighting it here might be a good way to do that. It might be better than expecting folks to periodically check a special run on team city. 

On the other hand it can?t be too noisy, else everyone will just filter it.

This is just an idea, feedback from any and all is welcome.

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat