[hibernate-dev] Analyzing the ORM testsuite

Thu Apr 19 20:14:02 EDT 2018

I checked - `build` does in fact pull in a bunch of tasks that you do not
care about (checkstyle, findbugs, etc).

The build I just ran did take longer than I have been seeing - it took 16
minutes, but that's not the magnitude you see.

I mentioned performing test caused javadoc to be generated.  I found the
culprit for that - hibernate-orm-modules.  Any time tests
use hibernate-orm-modules, hibernate-orm-modules triggers a full build and
publish of the project's artifacts (jar, source jar, javadoc jar).  This
accounts for at least some of the additional time.  I think using build
rather than test probably accounts for the rest

On Thu, Apr 19, 2018 at 5:35 PM Sanne Grinovero <sanne at hibernate.org> wrote:

> On 19 April 2018 at 22:50, Chris Cranford <chris at hibernate.org> wrote:
> > The entire build process under OpenJDK 8 here:
> >
> > BUILD SUCCESSFUL in 47m 26s
> > 251 actionable tasks: 233 executed, 18 up-to-date
> >
> > The only thing I noticed was that while executing the hibernate-core
> tests,
> > the memory consumption seemed quite high toward the end of the test
> suite,
> > particularly ~3.5GB memory consumed just to run the hibernate-core tests
> > alone and that FindBugs complained about some magic numbers on some zip
> > files while inspecting hibernate-osgi.
> >
> > Any idea if any substantial amount of that memory consumption related to
> the
> > test framework tracking the output/errors of those executed tests?
>
> I guess that confirms there's something specific to my environment.
> That's a great hint to have, thanks!
>
> Regarding the memory, yes I'm seeing the same. >90% of that is
> Mockito, I'm exploring some ideas to reduce its memory usage; I have
> some working experiments but I'm not happy with them: they make
> maintenance of tests error prone so tomorrow I'll try some
> alternatives.
>
>
> >
> > On 04/19/2018 05:20 PM, Sanne Grinovero wrote:
> >
> > On 19 April 2018 at 21:47, Chris Cranford <chris at hibernate.org> wrote:
> >
> > Sanne -
> >
> > Are you running the build task or are you executing other more specific
> > tasks and noticing the slowness?
> >
> > Just running "gradle clean build" from the root. I had the same
> > results with OpenJDK 8 and 9, twice. Getting exactly the same failures
> > as before.
> >
> > Thanks!
> >
> > On 04/19/2018 04:25 PM, Steve Ebersole wrote:
> >
> > Nothing off the top of my head.  The build time is in normal range on my
> > machine as of this morning (8 - 10 minutes).
> >
> > You mentioned a specific commit.  Is that just when you noticed a change,
> > or do you suspect something in that commit?
> >
> > I'm not at computer right now, but when I get back I will check out that
> > commit and see if anything is strange locally.
> >
> >
> >
> > On Thu, Apr 19, 2018, 3:11 PM Sanne Grinovero <sanne at hibernate.org>
> wrote:
> >
> > Hi all,
> >
> > I'm trying to analyze the memory usage of current master of Hibernate
> > ORM; I started looking because of OOM errors on the PostgreSQL
> > testsuite (as mentioned in another thread).
> >
> > So far I've seen that Mockito is allocating a "lot of stuff"; I might
> > be able to improve some things in that area but it's a distraction
> > from my goal as clearly we use Mockito even when not running the
> > PostgreSQL backed tests and that seems to run fine. But it's polluting
> > the data reports so making it hard (and slow!) to verify if there is a
> > real issue.
> >
> > I could use some help to verify some tangential issues though; when
> > running the testsuite locally it's taking much longer than what I've
> > been used to see for the same testsuite some weeks ago, and I have
> > some failures - regularly seeing the same failures.
> >
> > These are triggered by running a default build, on H2:
> >
> > Task :hibernate-core:test
> >
> > org.hibernate.test.criteria.CriteriaLockingTest >
> >
> >
> testSetLockModeDifferentFromNONELogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue
> > FAILED
> >     org.junit.runners.model.TestTimedOutException
> >
> > org.hibernate.test.criteria.CriteriaLockingTest >
> >
> >
> testSetLockModeNONEDoNotLogAWarnMessageWhenTheDialectUseFollowOnLockingIsTrue
> > FAILED
> >     java.lang.Exception
> >
> > org.hibernate.test.criteria.CriteriaLockingTest >
> > org.hibernate.test.criteria.CriteriaLockingTest FAILED
> >     java.lang.Exception
> >
> > org.hibernate.test.locking.warning.LockNoneWarmingTest >
> > org.hibernate.test.locking.warning.LockNoneWarmingTest FAILED
> >     java.lang.Exception
> >
> > 6307 tests completed, 4 failed, 327 skipped
> >
> > Yet the puzzling point: on ci.hibernate.org I don't see these failures
> > happening, nor the testsuite time seems to have increased of any
> > significant amount (it regularly completes in somewhere between 15 and
> > 20 minutes). It takes me more than an hour to get this result on a
> > machine which would normally complete the testsuite in 8 minutes, and
> > I got the same 4 failures when re-running it a second time.
> >
> > Could other developers also please checkout master - specifically at
> > commit 291d4a3eeaa2ade32a42cfbcad5868b2114c34fe - and let me know if
> > it works allright for you and if the build time is consistent with the
> > usual times for your machine? I need to understand what might be
> > different between my workstation and the CI server.
> >
> > I might even have some great improvements ready to reduce the
> > testsuite time, but I'm not comfortable in committing them in this
> > context.
> >
> > Thanks,
> > Sanne
> > _______________________________________________
> > hibernate-dev mailing list
> > hibernate-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >
> > _______________________________________________
> > hibernate-dev mailing list
> > hibernate-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >
> >
> > _______________________________________________
> > hibernate-dev mailing list
> > hibernate-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >
> >
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>