[hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

[Gail Badner]

I *think* that most, if not all, of the loaders for a particular entity
could be reworked to share most of the same information.

IIRC, the only real differences are the batch size and the actual query
that gets generated.

I'm happy to look into this, but I don't want to duplicate what others are
doing. It a bit difficult collaborating on this since our time zones don't
overlap very well (and, unfortunately, I'm not a morning person).

I have some other work I need to attend to today. Please let me know if I
should dig into this next week.

On Fri, May 4, 2018 at 12:30 PM, Guillaume Smet <guillaume.smet at gmail.com>
wrote:

> Don't know about the history but with the entity I checked out in the OP's
> case, each EntityLoader is approximately 200 KB.
>
> The entity contains a lot of columns so that's no surprise.
>
> What doesn't help is that I don't think the 13 EntityLoaders/entity share
> a lot of things so they are just adding up.
>
> On Fri, May 4, 2018 at 9:15 PM, Sanne Grinovero <sanne at hibernate.org>
> wrote:
>
>> On 4 May 2018 at 19:45, Gail Badner <gbadner at redhat.com> wrote:
>> > I don't think the batching strategies actually changed when we moved to
>> load
>> > plans.
>> >
>> > Is the problem that each loader is taking considerably more memory?
>>
>> Yes that's right.
>>
>> >
>> > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet <
>> guillaume.smet at gmail.com>
>> > wrote:
>> >>
>> >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero <sanne at hibernate.org>
>> >> wrote:
>> >>
>> >> > thanks for investigating. Steve in chat mentioned the option of
>> >> > creating these EntityLoaders lazily: as while they need to be reused,
>> >> > it's possible many of them are never really used.
>> >> >
>> >> > I think that's a very interesting observation, is that what you plan
>> to
>> >> > explore?
>> >> >
>> >>
>> >> Yes, it's one of the things we should explore.
>> >>
>> >>
>> >> > I see two difficulties:
>> >> >  - you'd need thredsafe code, yet we don't want barriers on this
>> area.
>> >> > Will probably need features such as lazy set, yet these would need
>> >> > even more objects allocated if you're not careful.
>> >> >  - we always prefer fail-at-boot validation, so we'd need to make
>> sure
>> >> > even if you don't build them all, we can at least make sure we'll be
>> >> > able to build them later.
>> >> >
>> >>
>> >> Well, the idea is to prototype something on the memory usage front,
>> then
>> >> we
>> >> can discuss further what we want to do with it and if it's worth
>> refining
>> >> it and spending more time on it.
>> >>
>> >> If we end up choosing this path, be sure we'll be bugging you for
>> advice
>> >> on
>> >> concurrency and performance :).
>> >>
>> >>
>> >> > Additionally there's the idea to see if any such Loader instance
>> could
>> >> > be reused - probably using a cache which we clear at end of bootstrap
>> >> > - yet since Steve mentioned this would be a significant change let's
>> >> > not try to push it in 5.3.
>> >> >
>> >> > Finally, a simple one: I have the impression that loader's Map could
>> >> > be made into a plain array, or a custom structure which would then be
>> >> > ideal to support lazy creation of loaders. With the exception of two
>> >> > special strings - which will needt to be treated specially - all
>> other
>> >> > keys are enums.
>> >> >
>> >>
>> >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet
>> sure
>> >> if
>> >> not knowing the code base will be a thing in my favor or not. At least
>> I
>> >> don't have preconceived ideas :].
>> >>
>> >>
>> >> > Whatever you choose to try, let's keep significant changes for 6
>> only :)
>> >> >
>> >>
>> >> Steve was saying that this whole thing wouldn't be an issue in 6.
>> >>
>> >> --
>> >> Guillaume
>> >> _______________________________________________
>> >> hibernate-dev mailing list
>> >> hibernate-dev at lists.jboss.org
>> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>> >
>> >
>> _______________________________________________
>> hibernate-dev mailing list
>> hibernate-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>
>
>