[hibernate-dev] Byte-buddy: default enhancer in 5.3 ? [HHH-11253]

[Sanne Grinovero]

An update on this; I've found multiple issues, yet when digging none
seem to be a strong reason to withold switching to Byte Buddy as
default.

The main problem seems to be that running the testsuite will somehow
load 2.867 different Class instances all having the same name
"org.hibernate.test.bytecode.enhancement.access.MixedAccessTest", and
a matching amount of
"org.hibernate.test.bytecode.enhancement.access.MixedAccessTest$NoDirtyCheckingContext"
classes.
I'm not really understanding how this happens yet, the ClassLoader
chain is huge and complex: there's a "root" classloader by Gradle
which applies some custom filtering, while on our test side this is
followed by two levels of enhancers implemented by two different
copies of byte-buddy: Mockito is including a shaded copy of byte-buddy
and has its own additional map with weak keys to cache more classes
and classloaders. So we're using Byte Buddy both for entity
enhancement, and to enhance our tests which I guess could include
re-enhancing the test entities in some situations.

We're also spending about half a gigabyte in empty HashMap instances,
and silly things such as 40MB in repeated copies of the same String or
same Integer instances. I could fix some little waste caused by
Hibernate ORM so the good news is that we'll be consuming a bit less
at runtime. Most of the duplication though is caused by overhead by
the huge amount of independent ClassLoaders: even class names are
repeated, package filtering and so on are repeated over and over into
the various loading caches of each classloader, repeated with similar
layers of Vector and LinkedHashMaps composing the repeated metadata
structures.

Now the good news: the test "MixedAccessTest" is marked with @Ignored.
If I delete it, everything works fine.

I don't like deleting things but I'm not fully understanding what this
test is meant to accomplish.. what shall I do? I propose we delete it,
anyone who's willing to get back to it can restore it from history.

I see three options:
 1) delete MixedAccessTest
 2) ignore it for now and raise the memory setting of the testsuite
from 2G to 8G
 3) keep working on this and see if I can fix the tests, hopefully
with some help

Thanks,
Sanne


On 28 February 2018 at 14:13, Steve Ebersole <steve at hibernate.org> wrote:
> I see a few (like in 3 or 4) Jiras mentioning people using Byte Buddy for
> enhancement, without Byte Buddy being the reason for the issue.  I cannot
> speak to how widely used it is.
>
> On Wed, Feb 28, 2018 at 7:10 AM Sanne Grinovero <sanne at hibernate.org> wrote:
>>
>> I found a couple more issues related to Byte Buddy which probably
>> should be fixed before we can make it the default.
>>
>> I've solved the trivial ones, but there's one more problem which I
>> couldn't nail down yet: the testsuite is using a significant amount of
>> additional memory when all tests default to the Byte Buddy based
>> enhancer: the requirements are rocketing from about 150MB to 4GB.
>>
>> A further complication is that for some reason Flight Recorder crashes
>> when connecting to the ORM testsuite; I've tried both versions coming
>> with JDK8 and JDK9 they both fail. I finally manged to get some
>> recordings from CLI tools so I hope to know more soon.
>>
>> I also noticed that the support for
>> hibernate.bytecode.provider=bytebuddy was not documented; that might
>> explain some lack of feedback on such issues. Makes me think that
>> maybe this wasn't tested much yet?
>>
>> Thanks,
>> Sanne
>>
>>
>> On 27 February 2018 at 11:13, Sanne Grinovero <sanne at hibernate.org> wrote:
>> > On 23 February 2018 at 20:29, Steve Ebersole <steve at hibernate.org>
>> > wrote:
>> >> You mean change to 5.3?  +1
>> >
>> > Right that's what I meant. Thanks, will do!
>> >
>> >>
>> >> On Fri, Feb 23, 2018 at 2:09 PM Sanne Grinovero <sanne at hibernate.org>
>> >> wrote:
>> >>>
>> >>> HHH-11253 is about making ByteBuddy the default implementation, but
>> >>> it's flagged for 6.
>> >>>
>> >>> I think that in Paris there was consensus to make it the default in
>> >>> 5.3 alredy, to possibly drop Javassist entirely in 6.0.
>> >>>
>> >>> May I change this to 6 ? If that's ok, I can assign it to myself.
>> >>>
>> >>> Thanks,
>> >>> Sanne
>> >>> _______________________________________________
>> >>> hibernate-dev mailing list
>> >>> hibernate-dev at lists.jboss.org
>> >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev