[hibernate-dev] About Buildhive, cloudbees?

[Sanne Grinovero]

Hi All,
I've had Buildhive configured on some of our projects for the last
week as an experiment (Hibernate Validator, Search, OGM).

Apologies for all the notifications it made, especially since I didn't
warn about enabling it.
Apparently it creates lots of false positives so it has been quite
noisy on all pull requests and commits.

It was super easy to setup, they definitely made an example to follow
in terms of service usability and UI: just login with your github
account, you can setup all projects you are admin of with a single
mouse click.

Now the bad news:
it's very limited, for example it wasn't able to build Hibernate ORM
as you can't choose the gradle version, and it's unable to run the
Infinispan testsuite as there is a 15 minutes build time limit.
Hibernate Search builds fails all Byteman related tests, I guess it's
missing the JDK's tools.jar from the classpath (not available in JRE).
OGM wasn't that bad, still occasionally it failed with apparently no
reason.

I've contacted Cloudbees to ask about the gradle version and I was
suggested to use their standard build platform, for which the same
pull-request-review plugin will be available soon, and which is much
more flexible in terms of configuration.
I think that's a reasonable suggestion and that would give us
most of the options we need, and we could get it for free as an open
source project.

It has never been my intention to replace JBoss's internal QA Jenkins
instances, nor I think that will be possible as only there we have all
different platforms to test on and all different databases, so I'm
exploring these options exclusively to get a filter between broken
pull requests and our reviews: if we can save some time and
efficiency, I think any additional help is welcome. Also some of the
tasks run by QA labs would be redundant - like most H2 run tests - and
we might save resources there for the other tasks by running a
selection of tests less frequently.

In conclusion: yesterday night I disabled it as I think it had way too
many false positives.

Shall we proceed in making an Hibernate account to setup some tasks on
the full-powered Cloudbees version? Again, not with the intention to
replace the role of "reference CI", but only to get some extra
processing power - especially the preventive tests on pull requests
are IMHO very nice.
I don't think it's a big cost as it's quite easy to configure and
maintain an additional set of Jenkins instances.

On a side note:
# I was looking into cloudbees anyway as they have free MongoDB
instances, so this would help in testing Hibernate OGM.
# I've tested openshift too for this purpose. Apparently the
expectation is that you have to commit "to" openshift directly to have
it trigger a test run: it's currently not able to monitor a different
git repository so that didn't seem very suited; it might be a good
place to run tests of demos to run on AS7 though.
Of course we might "push" the jenkins source code to it an use it as
was a self-made webapp, but then I think cloudbees would be more
effective as someone else will manage the platform.

Cheers,
Sanne