Jenkins job priorities
by Yoann Rodiere
Hello,
TL;DR: I installed a plugin to prioritize Jenkins jobs, please let me know
if you notice anything wrong. Also, I will remove the Heavy Job plugin
soon, let me know if you're not okay with that.
I recently raised the issue on HipChat that some Jenkins builds are
triggered in batch, something like 4 or 5 at a time. Since builds are
executed in the order they are requested, this forces the next requested
builds to wait for more than one hour before being executed, regardless of
their urgency.
One example of such batch is whenever something is pushed to Hibernate ORM
master (or Search master, probably): one build is triggered for tests
against H2, another for tests against PostgreSQL, another for tests against
MariaDB, and so on.
It turns out there is a solution for this problem: the PrioritySorter
plugin. I installed the plugin on CI and configured it to give higher
priority to the following builds:
- Builds triggered by users (highest priority)
- Release builds (builds in the "Release" view)
- Website builds (builds in the "Website" view)
- PR builds (builds in the "PR" view)
In practice, such builds will be moved to the front of the queue whenever
they are triggered, resulting in reduced waiting times.
I hope we will be able to use this priority feature instead of the Heavy
Job plugin (which allows to assign weights to jobs), and avoid concurrent
builds completely. With the current setup, someone releasing his/her
project will only have to wait for the currently executing build to finish,
and will get the highest priority on the release builds. Maybe this is
enough? If you disagree, please raise your concerns now: I will disable the
Heavy Job plugin soon and set each slave to only offer one execution slot.
Please let me know if you notice anything wrong. I tested the plugin on a
local Jenkins instance, but who knows...
Yoann
--
Yoann Rodiere
yoann(a)hibernate.org / yrodiere(a)redhat.com
Software Engineer
Hibernate NoORM team
8 years, 3 months
Bulk delete behavior for collection tables
by Christian Beikov
Hey,
so HHH-5529 <https://hibernate.atlassian.net/browse/HHH-5529> defines a
feature which I'd like to work on but want to hear opinions first.
Currently, bulk deletes only clear join tables of the affected entity
type. I guess one could argue that this was done because collection
table in contrast to join table entries should be bound to the entity
table lifecycle by using an FK with delete cascading. Or maybe it just
wasn't implemented because nobody stepped up.
I'd like to fill this gap and implement the deletion of the collection
table entries, but make that and the join table entry deletion configurable.
Does anyone have anything against that?
Would you prefer a single configuration option for join table and
collection table clearing? If we enable that option by default,
collection tables will then be cleared whereas currently users would get
a FK violation. Don't know if that can be classified as breaking behavior.
Or have two configuration options? Even then, would we enable collection
table entry deletion by default?
--
Mit freundlichen Grüßen,
------------------------------------------------------------------------
*Christian Beikov*
8 years, 3 months
Delay 5.3.0.Beta1 until next week
by Steve Ebersole
I am going to delay 5.3.0.Beta1 until next week to investigate OSSRH
publishing. I noticed yesterday that Bintray has a 10G storage limit which
we would hit too quickly.
Feel free to add issues to Beta1, but only if you plan on having them done
by next week (1/17).
8 years, 3 months
Awestruct upgrade to version 0.5.7
by Davide D'Alto
Hello,
I've upgraded awestruct to version 0.5.7.
Except for the minification of our stylesheets it seems to work fine
but It would be nice if someone else can have a look at generate site
and confirm to me that's
OK to apply the changes in production.
On the same note, I've noticed that we are using some custom
extensions to execute the minification, specifically the one in
_ext/css_minifier.rb.
Is there any reason to do that? I'm asking because Awestruct comes
with its own minification class called: Awestruct::Extension::Minify
The main difference I noticed is that this extension doesn't copy the
original file during deploy on the server and only uses the minified
one.
Also currently, this extension doesn't work for CSS but I guess they
are going to fix it in the next releases (it works for html and js).
Cheers,
Davide
8 years, 3 months
New CI slaves now available!
by Sanne Grinovero
Hi all,
we're having shiny new boxes running CI: more secure, way faster and
less "out of disk space" prolems I hope.
# Slaves
Slaves have been rebuilt from scratch:
- from Fedora 25 to Fedora 27
- NVMe disks for all storage, including databases, JDKs, dependency
stores, indexes and journals
- Now using C5 instances to benefit from Amazon's new "Nitro" engines [1]
- hardware offloading of network operations by enabling ENA [2]
- NVMe drives also using provisioned IO
This took a bit of unexpected low level work as .. Fedora images don't
support ENA yet so I had to create a custom Fedora re-distribution AMI
first, it wasn't possible to simply compile the kernel modules for the
standard Fedora images. These features are expected to come in future
Fedora Cloud images but I didn't want to wait so made our own :) [3]
# Cloud scaling
Idle slaves will self-terminate after some timeout (currently 30m).
When there are many jobs queueing up, more slaves (up to 5) will
automatically start.
If you're the first to trigger a build you'll have to be patient, as
it's possible after some quiet time (after the night?) all slaves are
gone; the system will boot up new ones automatically ASAP but this
initial boot takes some extra couple of minutes.
# Master node
Well, security patching mostly, but also finally figured out how to
workaround the bugs which were preventing us to upgrade Jenkins.
So now Jenkins is upgraded to latest, including *all plugins*. It
seems to work but let's keep an eye on it, those plugins are not all
maintained at the quality one would expect.
In particular attempting to change EC2 configuration properties will
now trigger a super annoying NPE [4]; either don't make further
changes or resort to XML editing of the configuration.
# Next
I'm not entirely done; eventually I'd like to convert our master node
to ENA/C5/NVMe as well - especially to be able to move all master and
slaves into the same physical cluster - but I'll stop now and get back
to Java so you all get a chance to identify problems caused by the new
slaves before I cause more trouble..
Thanks,
Sanne
1 - https://www.theregister.co.uk/2017/11/29/aws_reveals_nitro_architecture_b...
2 - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking-e...
3 - https://pagure.io/atomic-wg/issue/271
4 - https://issues.jenkins-ci.org/browse/JENKINS-46856
8 years, 3 months
CDI integration in Hibernate ORM and the Application scope
by Yoann Rodiere
Hello all,
TL;DR: Application-scoped beans cannot be used as part of the @PreDestroy
method of ORM-instantiated CDI beans, and it's a bit odd because they can
be used as part of the @PostConstruct method.
I've been testing the CDI integration in Hibernate ORM for the past few
days, trying to integrate it into Search. I think I've discovered something
odd: when CDI-managed beans are destroyed, they cannot access other
Application-scoped CDI beans anymore. Not sure whether this is a problem or
not, so maybe we should discuss it a bit before going forward with the
current behavior.
Short reminder: scopes define when CDI beans are created and destroyed.
@ApplicationScoped is pretty self-explanatory: created when the application
starts and destroyed when it stops. Some other scopes are a bit more
convoluted: @Singleton basically means created *before* the application
starts and destroyed *after* the application stops (and also means "this
bean shall not be proxied"), @Dependent means created when an instance is
requested and destroyed when the instance is released, etc.
The thing is, Hibernate ORM is typically started very early and shut down
very late in the CDI lifecycle - at least within WildFly. So when Hibernate
starts, CDI Application-scoped beans haven't been instantiated yet, and it
turns out that when Hibernate ORM shuts down, CDI has already destroyed
Application-scoped beans.
Regarding startup, Steve and Scott solved the problem by delaying bean
instantiation to some point in the future when the Application scope is
active (and thus Application-scoped beans are available). This makes it
possible to use Application-scoped beans within ORM-instantiated beans as
soon as the latter are constructed (i.e. within their @PostConstruct
methods).
However, when Hibernate ORM shuts down, the Application scope has already
been terminated. So when ORM destroys the beans it instantiated, those
ORM-instantiated beans cannot call a method on referenced
Application-scoped beans (CDI proxies will throw an exception).
All in all, the only type of beans we can currently use in a @PreDestroy
method of an ORM-instantiated bean is @Dependent beans. @Singleton beans
will work, but only because they are not proxied and thus you can cheat and
use them even after they have been destroyed... which I definitely wouldn't
recommend.
I see two ways to handle the issue:
1. We don't change anything, and simply document somewhere that beans
instantiated as part of the CDI integration are instantiated within the
Application scope, but are destroyed outside of it. And we suggest that any
bean used in @PostDestroy method in an ORM-instantiated bean (directly or
not) must have either a @Dependent scope, or a @Singleton scope and no
@PostDestroy method.
2. We implement an "early shut-down" somehow, which would bring forward
bean destruction to some time when the Application scope is still active.
#1 may be enough for now, even though the behavior feels a bit odd, and
forces users to resort to less-than-ideal practices (using a @Singleton
bean after it has been destroyed).
#2 would require changes in WildFly and may be a bit complex. In
particular, if we aren't careful, Application-scoped beans may not be able
to use Hibernate ORM from within their @PreDestroy methods... Which is
probably not a good idea. So we would have to find a solution together with
the WildFly team. Also to be considered: Hibernate Search would have to be
shut down just before the "early shut-down" of Hibernate ORM occurs,
because Hibernate Search cannot function at all without the beans it
retrieves from the CDI context.
Thoughts?
Yoann Rodière
Hibernate NoORM Team
yoann(a)hibernate.org
8 years, 3 months
