Can someone tell me why http://ci.hibernate.org/job/hibernate-orm-master/48/ is showing as failing? I looked in console and test output and see nothing (although it only runs the hibernate-c3p0 tests). _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

You mean for tests? Right now I see: test { ... maxHeapSize = "1024m" } On Sat 20 Apr 2013 02:20:56 PM CDT, Sanne Grinovero wrote: > Hi Steve, > from the Jenkins console I couldn't figure it out either; I looked on > the server and it seems the system kernel killed a java process > because it needed more memory. > > The system has plenty of memory available now that no tests are > running (about 1700MB), but from system logs it seems it regularly > kills some java ORM process; this is not a regular out-of-heap size so > it might be worth a short to revert the gradle memory options from > > maxHeapSize = "2048m" > > to > > maxHeapSize = "1024m" > > I had applied that change myself when we had the OOMs before because > of the leak we profiled ... > Also I'll keep in mind at the next machine refresh that we might want > more memory. > > Cheers, > Sanne > > > On 20 April 2013 17:24, Steve Ebersole <steve(a)hibernate.org&gt; wrote: >> Can someone tell me why >> http://ci.hibernate.org/job/hibernate-orm-master/48/ is showing as >> failing? >> >> I looked in console and test output and see nothing (although it only >> runs the hibernate-c3p0 tests). >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev

This seems to be tied to the commits for https://hibernate.atlassian.net/browse/HHH-8127 Strong, these were your commits. Can you give us any insight here? The ci.hibernate.org tests were passing before your work : http://ci.hibernate.org/job/hibernate-orm-master/52/ They started failing after your commit, and have not completed since : http://ci.hibernate.org/job/hibernate-orm-master/53/ On Tue 23 Apr 2013 06:10:10 PM CDT, Steve Ebersole wrote: > Any other thoughts one what to do here? > > Sanne, not sure if you saw my earlier response, but heap size is > already set to 1024m, not 2048m. The only change I pulled over from > that was to add jvmArgs '-XX:MaxPermSize=256m' as well. Do you want > me to back that out as well? > > > On Mon 22 Apr 2013 05:01:08 PM CDT, Steve Ebersole wrote: >> You mean for tests? Right now I see: >> >> test { >> ... >> maxHeapSize = "1024m" >> } >> >> >> On Sat 20 Apr 2013 02:20:56 PM CDT, Sanne Grinovero wrote: >>> Hi Steve, >>> from the Jenkins console I couldn't figure it out either; I looked on >>> the server and it seems the system kernel killed a java process >>> because it needed more memory. >>> >>> The system has plenty of memory available now that no tests are >>> running (about 1700MB), but from system logs it seems it regularly >>> kills some java ORM process; this is not a regular out-of-heap size so >>> it might be worth a short to revert the gradle memory options from >>> >>> maxHeapSize = "2048m" >>> >>> to >>> >>> maxHeapSize = "1024m" >>> >>> I had applied that change myself when we had the OOMs before because >>> of the leak we profiled ... >>> Also I'll keep in mind at the next machine refresh that we might want >>> more memory. >>> >>> Cheers, >>> Sanne >>> >>> >>> On 20 April 2013 17:24, Steve Ebersole <steve(a)hibernate.org&gt; wrote: >>>> Can someone tell me why >>>> http://ci.hibernate.org/job/hibernate-orm-master/48/ is showing as >>>> failing? >>>> >>>> I looked in console and test output and see nothing (although it only >>>> runs the hibernate-c3p0 tests). >>>> _______________________________________________ >>>> hibernate-dev mailing list >>>> hibernate-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

Sanne, is it possible that you give me ssh access to the ci server?

As far as I can tell there are no tests leaving the SessionFactory open in hibernate-core. I have not tested yet with the other modules, but from what I can see the CI builds fail at the point trying to run these tests. Essentially I added a hook in CustomRunner (in AfterClassCallbackHandler to be precise) to check the SessionFactoryRegistry to make sure there are no still registered SessionFactories after a test class is done. If there are, it logs a message and clears the SessionFactoryRegistry. This is simplistic as we still have other resources that need to be checked (ServiceRegistries are used independently from SessionFactory quite a bit). This was after I got all tests indicated by Strong to extend BaseUnitTestCase that did not previously. I did have problems with org.hibernate.metamodel.**source.annotations.xml.mocker.**AbstractMockerTest and subclasses. Having it extend BaseUnitTestCase with that functionality led to assertion failures in 3 tests: BasicMockerTest.**testEntityWithEntityMappingsCo**nfiguration OverrideTest.**testPersistenceUnitDefaultsCas**cadePersistInAnnotation OverrideTest.**testPersistenceUnitMetadataMet**adataComplete Which is odd, because those tests don't even deal with a SessionFactory. As for checking that created ServiceRegistry instances get cleaned up, not sure how we can check that. Another thing we should do is to exclude the metamodel tests currently on master. After that, I am out of ideas... On Thu 25 Apr 2013 05:28:35 AM CDT, Sanne Grinovero wrote: > On 25 April 2013 09:04, Strong Liu <stliu(a)hibernate.org&gt; wrote: > >> Sanne, >> >> is it possible that you give me ssh access to the ci server? >> > > Sure; I'm adding Strong, if anyone else wants root access just send me > your public key using your redhat account. > > Cheers, > Sanne >

I am watching a build I just kicked off right now in the ui console and the sticking point seems to actually be compiling the test classes (unless I am mis-understanding the console). The step where the build hung (just before dieing) was: :hibernate-core:testClasses I have to see what all testClasses involves. Actual test source compilation was already completed earlier. On Thu 25 Apr 2013 03:32:30 PM CDT, Sanne Grinovero wrote: > > I think it's an environment problem of our CI, I yet have to > understand what this message means but while using less memory could > be a valid workaround, I still suspect the problem is not in > Hibernate's code. Sorry this is going to be a bit slow I need to get a > RH subscription to get proper support.. Suggestion from Linux experts > welcome in the meantime: I've never seen such an error before. > > Sanne > > On 25 Apr 2013 20:53, "Steve Ebersole" <steve(a)hibernate.org > <mailto:steve@hibernate.org>> wrote: > > As far as I can tell there are no tests leaving the SessionFactory > open in hibernate-core. I have not tested yet with the other > modules, but from what I can see the CI builds fail at the point > trying to run these tests. > > Essentially I added a hook in CustomRunner (in > AfterClassCallbackHandler to be precise) to check the > SessionFactoryRegistry to make sure there are no still registered > SessionFactories after a test class is done. If there are, it > logs a message and clears the SessionFactoryRegistry. This is > simplistic as we still have other resources that need to be > checked (ServiceRegistries are used independently from > SessionFactory quite a bit). > > This was after I got all tests indicated by Strong to extend > BaseUnitTestCase that did not previously. I did have problems > with > > org.hibernate.metamodel.__source.annotations.xml.mocker.__AbstractMockerTest > > and subclasses. Having it extend BaseUnitTestCase with that > functionality led to assertion failures in 3 tests: > > BasicMockerTest.__testEntityWithEntityMappingsCo__nfiguration > OverrideTest.__testPersistenceUnitDefaultsCas__cadePersistInAnnotation > OverrideTest.__testPersistenceUnitMetadataMet__adataComplete > > > Which is odd, because those tests don't even deal with a > SessionFactory. > > As for checking that created ServiceRegistry instances get cleaned > up, not sure how we can check that. > > Another thing we should do is to exclude the metamodel tests > currently on master. After that, I am out of ideas... > > On Thu 25 Apr 2013 05:28:35 AM CDT, Sanne Grinovero wrote: > > On 25 April 2013 09:04, Strong Liu <stliu(a)hibernate.org > <mailto:stliu@hibernate.org>> wrote: > > Sanne, > > is it possible that you give me ssh access to the ci server? > > > Sure; I'm adding Strong, if anyone else wants root access just > send me > your public key using your redhat account. > > Cheers, > Sanne >

On the new machine I am not able to get tests hooked in properly. I added a "post build hook" to "publish test results", but the build fails saying it does not like my fileset definition. I set it as "**/target/test-results/*.xml", but it says it is not able to find any tests using that. Do we maybe need a special plugin for aggregated multi-module test result publishing? Even worse, after enabling these post-build actions the build time has actually increased pretty significantly. It went from approx 21 minutes to run the full build to the last build taking 38 minutes and that does not even include (1) the completion of the test result publishing and (2) the snapshot artifact publishing to nexus. This might be the fulcrum point for the discussion about separate triggered and nightly builds. On Thu 25 Apr 2013 06:46:12 PM CDT, Sanne Grinovero wrote: > I got it to work with a filthy hack: I've created a new virtual > volume, attached it to the VM and activated it as additional swap > drive.. > > This way the build wasn't killed and could complete, however (as > expected) it took ages to complete. > > At least now we know the proper remedy: use a machine with more than > 2GB or RAM; I had underestimated our needs, luckily it's an easy fix: > tomorrow I'll transfer the VM into a different host. > > http://ci.hibernate.org/job/hibernate-orm-master/72/ > > Sanne > > > > On 25 April 2013 23:49, Steve Ebersole <steve(a)hibernate.org&gt; wrote: >> I am watching a build I just kicked off right now in the ui console >> and the >> sticking point seems to actually be compiling the test classes >> (unless I am >> mis-understanding the console). >> >> The step where the build hung (just before dieing) was: >> :hibernate-core:testClasses >> >> I have to see what all testClasses involves. Actual test source >> compilation >> was already completed earlier. >> >> >> >> On Thu 25 Apr 2013 03:32:30 PM CDT, Sanne Grinovero wrote: >>> >>> I think it's an environment problem of our CI, I yet have to >>> understand what this message means but while using less memory could >>> be a valid workaround, I still suspect the problem is not in >>> Hibernate's code. Sorry this is going to be a bit slow I need to get a >>> RH subscription to get proper support.. Suggestion from Linux experts >>> welcome in the meantime: I've never seen such an error before. >>> >>> Sanne >>> >>> On 25 Apr 2013 20:53, "Steve Ebersole" <steve(a)hibernate.org >>> <mailto:steve@hibernate.org>> wrote: >>> >>> As far as I can tell there are no tests leaving the >>> SessionFactory >>> open in hibernate-core. I have not tested yet with the other >>> modules, but from what I can see the CI builds fail at the point >>> trying to run these tests. >>> >>> Essentially I added a hook in CustomRunner (in >>> AfterClassCallbackHandler to be precise) to check the >>> SessionFactoryRegistry to make sure there are no still registered >>> SessionFactories after a test class is done. If there are, it >>> logs a message and clears the SessionFactoryRegistry. This is >>> simplistic as we still have other resources that need to be >>> checked (ServiceRegistries are used independently from >>> SessionFactory quite a bit). >>> >>> This was after I got all tests indicated by Strong to extend >>> BaseUnitTestCase that did not previously. I did have problems >>> with >>> >>> org.hibernate.metamodel.__source.annotations.xml.mocker.__AbstractMockerTest >>> >>> >>> and subclasses. Having it extend BaseUnitTestCase with that >>> functionality led to assertion failures in 3 tests: >>> >>> BasicMockerTest.__testEntityWithEntityMappingsCo__nfiguration >>> >>> OverrideTest.__testPersistenceUnitDefaultsCas__cadePersistInAnnotation >>> OverrideTest.__testPersistenceUnitMetadataMet__adataComplete >>> >>> >>> Which is odd, because those tests don't even deal with a >>> SessionFactory. >>> >>> As for checking that created ServiceRegistry instances get >>> cleaned >>> up, not sure how we can check that. >>> >>> Another thing we should do is to exclude the metamodel tests >>> currently on master. After that, I am out of ideas... >>> >>> On Thu 25 Apr 2013 05:28:35 AM CDT, Sanne Grinovero wrote: >>> >>> On 25 April 2013 09:04, Strong Liu <stliu(a)hibernate.org >>> <mailto:stliu@hibernate.org>> wrote: >>> >>> Sanne, >>> >>> is it possible that you give me ssh access to the ci >>> server? >>> >>> >>> Sure; I'm adding Strong, if anyone else wants root access >>> just >>> send me >>> your public key using your redhat account. >>> >>> Cheers, >>> Sanne >>> >>