[hibernate-dev] About Buildhive, cloudbees?

Hi All, I've had Buildhive configured on some of our projects for the last week as an experiment (Hibernate Validator, Search, OGM). Apologies for all the notifications it made, especially since I didn't warn about enabling it. Apparently it creates lots of false positives so it has been quite noisy on all pull requests and commits. It was super easy to setup, they definitely made an example to follow in terms of service usability and UI: just login with your github account, you can setup all projects you are admin of with a single mouse click. Now the bad news: it's very limited, for example it wasn't able to build Hibernate ORM as you can't choose the gradle version, and it's unable to run the Infinispan testsuite as there is a 15 minutes build time limit. Hibernate Search builds fails all Byteman related tests, I guess it's missing the JDK's tools.jar from the classpath (not available in JRE). OGM wasn't that bad, still occasionally it failed with apparently no reason. I've contacted Cloudbees to ask about the gradle version and I was suggested to use their standard build platform, for which the same pull-request-review plugin will be available soon, and which is much more flexible in terms of configuration. I think that's a reasonable suggestion and that would give us most of the options we need, and we could get it for free as an open source project. It has never been my intention to replace JBoss's internal QA Jenkins instances, nor I think that will be possible as only there we have all different platforms to test on and all different databases, so I'm exploring these options exclusively to get a filter between broken pull requests and our reviews: if we can save some time and efficiency, I think any additional help is welcome. Also some of the tasks run by QA labs would be redundant - like most H2 run tests - and we might save resources there for the other tasks by running a selection of tests less frequently. In conclusion: yesterday night I disabled it as I think it had way too many false positives. Shall we proceed in making an Hibernate account to setup some tasks on the full-powered Cloudbees version? Again, not with the intention to replace the role of "reference CI", but only to get some extra processing power - especially the preventive tests on pull requests are IMHO very nice. I don't think it's a big cost as it's quite easy to configure and maintain an additional set of Jenkins instances. On a side note: # I was looking into cloudbees anyway as they have free MongoDB instances, so this would help in testing Hibernate OGM. # I've tested openshift too for this purpose. Apparently the expectation is that you have to commit "to" openshift directly to have it trigger a test run: it's currently not able to monitor a different git repository so that didn't seem very suited; it might be a good place to run tests of demos to run on AS7 though. Of course we might "push" the jenkins source code to it an use it as was a self-made webapp, but then I think cloudbees would be more effective as someone else will manage the platform. Cheers, Sanne