[hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

[Guillaume Smet]

As already mentioned, I plan to take a look at this next week.

I also thought we could share most of the info between loaders but Steve
was thinking it might not be possible without massive changes.

I'll take a look and report back to all of you. I might be way over my head
as I don't know the codebase very well but I already did a very similar
work for HV with great results so it's worth a shot.

-- 
Guillaume

On Fri, May 4, 2018 at 10:31 PM, Gail Badner <gbadner at redhat.com> wrote:

> I *think* that most, if not all, of the loaders for a particular entity
> could be reworked to share most of the same information.
>
> IIRC, the only real differences are the batch size and the actual query
> that gets generated.
>
> I'm happy to look into this, but I don't want to duplicate what others are
> doing. It a bit difficult collaborating on this since our time zones don't
> overlap very well (and, unfortunately, I'm not a morning person).
>
> I have some other work I need to attend to today. Please let me know if I
> should dig into this next week.
>
> On Fri, May 4, 2018 at 12:30 PM, Guillaume Smet <guillaume.smet at gmail.com>
> wrote:
>
>> Don't know about the history but with the entity I checked out in the
>> OP's case, each EntityLoader is approximately 200 KB.
>>
>> The entity contains a lot of columns so that's no surprise.
>>
>> What doesn't help is that I don't think the 13 EntityLoaders/entity share
>> a lot of things so they are just adding up.
>>
>> On Fri, May 4, 2018 at 9:15 PM, Sanne Grinovero <sanne at hibernate.org>
>> wrote:
>>
>>> On 4 May 2018 at 19:45, Gail Badner <gbadner at redhat.com> wrote:
>>> > I don't think the batching strategies actually changed when we moved
>>> to load
>>> > plans.
>>> >
>>> > Is the problem that each loader is taking considerably more memory?
>>>
>>> Yes that's right.
>>>
>>> >
>>> > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet <
>>> guillaume.smet at gmail.com>
>>> > wrote:
>>> >>
>>> >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero <sanne at hibernate.org
>>> >
>>> >> wrote:
>>> >>
>>> >> > thanks for investigating. Steve in chat mentioned the option of
>>> >> > creating these EntityLoaders lazily: as while they need to be
>>> reused,
>>> >> > it's possible many of them are never really used.
>>> >> >
>>> >> > I think that's a very interesting observation, is that what you
>>> plan to
>>> >> > explore?
>>> >> >
>>> >>
>>> >> Yes, it's one of the things we should explore.
>>> >>
>>> >>
>>> >> > I see two difficulties:
>>> >> >  - you'd need thredsafe code, yet we don't want barriers on this
>>> area.
>>> >> > Will probably need features such as lazy set, yet these would need
>>> >> > even more objects allocated if you're not careful.
>>> >> >  - we always prefer fail-at-boot validation, so we'd need to make
>>> sure
>>> >> > even if you don't build them all, we can at least make sure we'll be
>>> >> > able to build them later.
>>> >> >
>>> >>
>>> >> Well, the idea is to prototype something on the memory usage front,
>>> then
>>> >> we
>>> >> can discuss further what we want to do with it and if it's worth
>>> refining
>>> >> it and spending more time on it.
>>> >>
>>> >> If we end up choosing this path, be sure we'll be bugging you for
>>> advice
>>> >> on
>>> >> concurrency and performance :).
>>> >>
>>> >>
>>> >> > Additionally there's the idea to see if any such Loader instance
>>> could
>>> >> > be reused - probably using a cache which we clear at end of
>>> bootstrap
>>> >> > - yet since Steve mentioned this would be a significant change let's
>>> >> > not try to push it in 5.3.
>>> >> >
>>> >> > Finally, a simple one: I have the impression that loader's Map could
>>> >> > be made into a plain array, or a custom structure which would then
>>> be
>>> >> > ideal to support lazy creation of loaders. With the exception of two
>>> >> > special strings - which will needt to be treated specially - all
>>> other
>>> >> > keys are enums.
>>> >> >
>>> >>
>>> >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet
>>> sure
>>> >> if
>>> >> not knowing the code base will be a thing in my favor or not. At
>>> least I
>>> >> don't have preconceived ideas :].
>>> >>
>>> >>
>>> >> > Whatever you choose to try, let's keep significant changes for 6
>>> only :)
>>> >> >
>>> >>
>>> >> Steve was saying that this whole thing wouldn't be an issue in 6.
>>> >>
>>> >> --
>>> >> Guillaume
>>> >> _______________________________________________
>>> >> hibernate-dev mailing list
>>> >> hibernate-dev at lists.jboss.org
>>> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>> >
>>> >
>>> _______________________________________________
>>> hibernate-dev mailing list
>>> hibernate-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>>
>>
>>
>