I was curious why it took so long to run the master ORM jobs on the CI machine compared to running the job locally. Locally I run `clean test` at the root prject quite often and it takes roughly 9-10 minutes. The master CI jobs generally take 45-50 minutes to complete. So I enabled "Gradle build profiling" in our job. The results were surprising in terms of ratios. I figured findBugs, checkStyle etc probably added significant times to the build. But I was shocked how much it added. BTW, you can view these profile reports in {root}/build/reports/profile... So hibernate-core, overall took 17m22.19s to run for one job. Of that, 12.5 was findBugs! checkStyle as actually "reasonable" at just under 30s. The ratios were similar across all modules. The aggregatedJavadoc task took a shade over 2m. Considering that these jobs are run on ever check-in (and eventually it would be great to auto-run them against PRs too), plus the fact that we aren't even failing the build for the majority of findBug/checkStyle hits I think we should define these jobs a little differently. Its not just the time it takes. Yes we all hate to wait. But it's also the CI resources taken up. I'm going to put some thought into this after the 5.0 Beta release, but I wanted to get some thoughts and feedback in the meantime. Things to consider. _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

We should remove that -XX:MaxPermSize=512m setting as we discussed before.

I understand all of that. I still think its probably worthwhile to look at minimizing the amount of time and resources. For a couple of reasons. First I am doing a lot of dev right now. In the 40-50 minutes it takes to run the CI job after a push I often have other pushes queued up. Then when that first jobs fails, I need to go back and shift gears back into that first issue again. And spending roughly 75-80% of that time in a task that we don't even fail the build for seems excessive. My preliminary thinking here is to define a more than one job. Setting aside the idea of db-jobs for the moment... 1) Per-push job - triggered by push a) runs tests b) simplified checks (findbugs, checkstyle) that we actually fail the build for c) publish 2) Nightly build - triggered nightly (only with change in source?) a) full checks (both failures and warnings) b) aggregated javadocs * I'd actually like the nightly build to trigger once a day when a push actually occured. Not sure that is possible.

The "per-push" job really is a "main" one meant to run against H2. We still need to set up the db specific jobs:

3) DB job - triggered on successful execution of (1)? a) run tests And ultimately I'd love to setup some jobs for JDK testing that pull the artifacts built by (1) and tests them in a different JDK. Currently we need the Java 8 JDK to do a full build of ORM (JSR 310 support). For the modules other than hibernate-java8 we set source and target compatibility to Java 6. We could simply set up some jobs that use Java 6 JDK for the build, but that's not quite the same as pulling the artifacts built using Java 8 and targetting Java 6 and trying to run those in Java 6 or 7 environment. I don't know if Gradle has any existing support for this or whether this is another plugin I will have to develop to make that happen.

On Fri, Mar 27, 2015 at 4:38 PM, Sanne Grinovero <sanne(a)hibernate.org&gt; wrote: > > Hi Steve, > don't worry for the resources they consume. Each build job which we > run is fully isolated and running on independent hardware, so it's not > going to slow down any other build. > > Since we now have a set of small servers, one each dedicated for a job > at a time, I've opted for low spec machines as an experiment. This > performance is however disappointing, we can try different types of > CPU/Memory/Disk categories. > > The current specs of each build node are: > 2 CPUs > 3 GB of RAM > 18GB of root disk (of which 9GB available for our builds) > 50GB available to run various docker images and temporary databases > (allocated per job need) > > They run Fedora 21, Cloud Edition. > > As you can see, resources on a single build node are very limited - > especially space and memory. > But since there is no swap space configured (!) I expect that in case > the memory is not enough, it will crash (and therefore let us know) > rather than slow down. > It's possible though that since the ORM builds allocate 2GB of heap, > we're preventing the operating system from doing proper filesystem > caching. > > currently we have: > GRADLE_OPTS="-Xmx2G -Xms2G -XX:MaxPermSize=512m" > > I'm going to try to lower that, bear with me if there are failures. If > that doesn't work we can definitely replace these with larger > instances, it should take us just some minutes as it's all automated: > the only reason for me to be conservative is to make fair use of these > (free) resource, and also I'd like us to actually test for not needing > much ram - we can get there in smaller steps as needed of course. > > Sanne > > On 27 March 2015 at 21:01, Steve Ebersole <steve(a)hibernate.org&gt; wrote: > > I was curious why it took so long to run the master ORM jobs on the CI > > machine compared to running the job locally. Locally I run `clean test` > > at > > the root prject quite often and it takes roughly 9-10 minutes. The > > master > > CI jobs generally take 45-50 minutes to complete. > > > > So I enabled "Gradle build profiling" in our job. The results were > > surprising in terms of ratios. > > > > I figured findBugs, checkStyle etc probably added significant times to > > the > > build. But I was shocked how much it added. > > > > BTW, you can view these profile reports in > > {root}/build/reports/profile... > > > > So hibernate-core, overall took 17m22.19s to run for one job. Of that, > > 12.5 was findBugs! checkStyle as actually "reasonable" at just under > > 30s. > > The ratios were similar across all modules. > > > > The aggregatedJavadoc task took a shade over 2m. > > > > Considering that these jobs are run on ever check-in (and eventually it > > would be great to auto-run them against PRs too), plus the fact that we > > aren't even failing the build for the majority of findBug/checkStyle > > hits I > > think we should define these jobs a little differently. Its not just > > the > > time it takes. Yes we all hate to wait. But it's also the CI resources > > taken up. > > > > I'm going to put some thought into this after the 5.0 Beta release, but > > I > > wanted to get some thoughts and feedback in the meantime. Things to > > consider. > > _______________________________________________ > > hibernate-dev mailing list > > hibernate-dev(a)lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hibernate-dev

On Sat, Mar 28, 2015 at 11:26 AM, Sanne Grinovero <sanne(a)hibernate.org&gt; wrote: > > Consider also that since we now have N slaves, there is no guarantee > that a second job will run on the same build slave. > So if the JDK8 build produces artifacts but only deployes them in the > local repository, the second node might not find the same as it has an > independent repository cache. > This is one of the major differences between Gradle and Maven. Gradle has no idea about a "local repository" in terms of publishing. These jobs, if they publish, always publish to Nexus. We added the ability to read artifacts from "Maven local" mainly to deal with cases where we have local Maven-built projects that we want to have ORM pick up a local snapshot of. HCANN was the main thing here. But the "Gradle way" is to have the "other" local project (e.g., HCANN) publish its SNAPSHOT to Nexus and for the ORM build to pick it up from there. More repeatable builds that way. You pointed out one such scenario. Of course locally, Gradle (with some help usually) also lets you aggregate these disparate projects together into one cohesive build. And given that IIRC I converted HCANN to use Gradle for building as well, maybe its time to pull that ability. One way would be to actually upload the snapshots to the jboss Nexus, > but it could be easier / more efficient to have a single job which > does the compilation & packaging only with JDK8 but then runs the > tests with JDK6 ? > The JDK paths are well defined, so that should be scriptable with a > couple of sequential invocations to Gradle? That is another option for sure. And actually it does not even have to be separate invocations. We could script the creation of a new Test task based on the configured JDK6 path and just have both run in the same Gradle build.

Does Eclipse have the ability, like IntelliJ does, to define a different JDK for each module in a multi-module project? On Sat, Mar 28, 2015 at 2:10 PM, Steve Ebersole <steve(a)hibernate.org&gt; wrote: > Gradle really is quite elegant in most cases: > > if ( project.name != 'hibernate-java8' && rootProject.hasProperty( > "JDK6_HOME" ) ) { > final File java6Home = new File( rootProject.property( "JDK6_HOME" ) as > String ); > final org.gradle.internal.jvm.Jvm java6Jdk = > org.gradle.internal.jvm.Jvm.forHome( java6Home ) as > org.gradle.internal.jvm.Jvm; > if ( !java6Jdk.javaVersion.java6 ) { > throw new GradleException( "JDK6_HOME (if set) must point to a Java 1.6 > home, found ${java6Jdk.javaVersion}" ); > } > final Test mainTestTask = subProject.tasks.test as Test; > task testWithJdk6(type:Test) { > inputs = mainTestTask.inputs > outputs = mainTestTask.outputs > > if ( java6Jdk.toolsJar ) { > bootstrapClasspath java6Jdk.toolsJar > } > > classpath = java6Jdk.runtimeJar + mainTestTask.classpath > executable = java6Jdk.javaExecutable > > // should we recompile the tests using JDK6? If so, set a different > classes dir > // here and clone new JavaCompile task for tests. But just running with > the Java 6 > // JRE should be enough > testClassesDir = mainTestTask.testClassesDir > testSrcDirs = mainTestTask.testSrcDirs > > enableAssertions = mainTestTask.enableAssertions > environment = mainTestTask.environment > systemProperties = mainTestTask.systemProperties > scanForTestClasses = mainTestTask.scanForTestClasses > includes = mainTestTask.includes > excludes = mainTestTask.excludes > jvmArgs = mainTestTask.jvmArgs > maxHeapSize = mainTestTask.maxHeapSize > minHeapSize = mainTestTask.minHeapSize > } > } > > On Sat, Mar 28, 2015 at 11:46 AM, Steve Ebersole <steve(a)hibernate.org&gt; > wrote: > >> On Sat, Mar 28, 2015 at 11:26 AM, Sanne Grinovero <sanne(a)hibernate.org&gt; >> wrote: >>> >>> Consider also that since we now have N slaves, there is no guarantee >>> that a second job will run on the same build slave. >>> So if the JDK8 build produces artifacts but only deployes them in the >>> local repository, the second node might not find the same as it has an >>> independent repository cache. >>> >> >> This is one of the major differences between Gradle and Maven. Gradle >> has no idea about a "local repository" in terms of publishing. These jobs, >> if they publish, always publish to Nexus. >> >> We added the ability to read artifacts from "Maven local" mainly to deal >> with cases where we have local Maven-built projects that we want to have >> ORM pick up a local snapshot of. HCANN was the main thing here. But the >> "Gradle way" is to have the "other" local project (e.g., HCANN) publish its >> SNAPSHOT to Nexus and for the ORM build to pick it up from there. >> >> More repeatable builds that way. You pointed out one such scenario. Of >> course locally, Gradle (with some help usually) also lets you aggregate >> these disparate projects together into one cohesive build. >> >> And given that IIRC I converted HCANN to use Gradle for building as >> well, maybe its time to pull that ability. >> >> >> One way would be to actually upload the snapshots to the jboss Nexus, >>> but it could be easier / more efficient to have a single job which >>> does the compilation & packaging only with JDK8 but then runs the >>> tests with JDK6 ? >>> The JDK paths are well defined, so that should be scriptable with a >>> couple of sequential invocations to Gradle? >> >> >> That is another option for sure. And actually it does not even have to >> be separate invocations. We could script the creation of a new Test task >> based on the configured JDK6 path and just have both run in the same Gradle >> build. >> >> > >