[hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

[Gail Badner]

OK, that's fine. I did a lot of the work converting to load plans, so feel
free to ask if you have questions.

On Fri, May 4, 2018 at 1:37 PM, Guillaume Smet <guillaume.smet at gmail.com>
wrote:

> As already mentioned, I plan to take a look at this next week.
>
> I also thought we could share most of the info between loaders but Steve
> was thinking it might not be possible without massive changes.
>
> I'll take a look and report back to all of you. I might be way over my
> head as I don't know the codebase very well but I already did a very
> similar work for HV with great results so it's worth a shot.
>
> --
> Guillaume
>
> On Fri, May 4, 2018 at 10:31 PM, Gail Badner <gbadner at redhat.com> wrote:
>
>> I *think* that most, if not all, of the loaders for a particular entity
>> could be reworked to share most of the same information.
>>
>> IIRC, the only real differences are the batch size and the actual query
>> that gets generated.
>>
>> I'm happy to look into this, but I don't want to duplicate what others
>> are doing. It a bit difficult collaborating on this since our time zones
>> don't overlap very well (and, unfortunately, I'm not a morning person).
>>
>> I have some other work I need to attend to today. Please let me know if I
>> should dig into this next week.
>>
>> On Fri, May 4, 2018 at 12:30 PM, Guillaume Smet <guillaume.smet at gmail.com
>> > wrote:
>>
>>> Don't know about the history but with the entity I checked out in the
>>> OP's case, each EntityLoader is approximately 200 KB.
>>>
>>> The entity contains a lot of columns so that's no surprise.
>>>
>>> What doesn't help is that I don't think the 13 EntityLoaders/entity
>>> share a lot of things so they are just adding up.
>>>
>>> On Fri, May 4, 2018 at 9:15 PM, Sanne Grinovero <sanne at hibernate.org>
>>> wrote:
>>>
>>>> On 4 May 2018 at 19:45, Gail Badner <gbadner at redhat.com> wrote:
>>>> > I don't think the batching strategies actually changed when we moved
>>>> to load
>>>> > plans.
>>>> >
>>>> > Is the problem that each loader is taking considerably more memory?
>>>>
>>>> Yes that's right.
>>>>
>>>> >
>>>> > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet <
>>>> guillaume.smet at gmail.com>
>>>> > wrote:
>>>> >>
>>>> >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero <
>>>> sanne at hibernate.org>
>>>> >> wrote:
>>>> >>
>>>> >> > thanks for investigating. Steve in chat mentioned the option of
>>>> >> > creating these EntityLoaders lazily: as while they need to be
>>>> reused,
>>>> >> > it's possible many of them are never really used.
>>>> >> >
>>>> >> > I think that's a very interesting observation, is that what you
>>>> plan to
>>>> >> > explore?
>>>> >> >
>>>> >>
>>>> >> Yes, it's one of the things we should explore.
>>>> >>
>>>> >>
>>>> >> > I see two difficulties:
>>>> >> >  - you'd need thredsafe code, yet we don't want barriers on this
>>>> area.
>>>> >> > Will probably need features such as lazy set, yet these would need
>>>> >> > even more objects allocated if you're not careful.
>>>> >> >  - we always prefer fail-at-boot validation, so we'd need to make
>>>> sure
>>>> >> > even if you don't build them all, we can at least make sure we'll
>>>> be
>>>> >> > able to build them later.
>>>> >> >
>>>> >>
>>>> >> Well, the idea is to prototype something on the memory usage front,
>>>> then
>>>> >> we
>>>> >> can discuss further what we want to do with it and if it's worth
>>>> refining
>>>> >> it and spending more time on it.
>>>> >>
>>>> >> If we end up choosing this path, be sure we'll be bugging you for
>>>> advice
>>>> >> on
>>>> >> concurrency and performance :).
>>>> >>
>>>> >>
>>>> >> > Additionally there's the idea to see if any such Loader instance
>>>> could
>>>> >> > be reused - probably using a cache which we clear at end of
>>>> bootstrap
>>>> >> > - yet since Steve mentioned this would be a significant change
>>>> let's
>>>> >> > not try to push it in 5.3.
>>>> >> >
>>>> >> > Finally, a simple one: I have the impression that loader's Map
>>>> could
>>>> >> > be made into a plain array, or a custom structure which would then
>>>> be
>>>> >> > ideal to support lazy creation of loaders. With the exception of
>>>> two
>>>> >> > special strings - which will needt to be treated specially - all
>>>> other
>>>> >> > keys are enums.
>>>> >> >
>>>> >>
>>>> >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet
>>>> sure
>>>> >> if
>>>> >> not knowing the code base will be a thing in my favor or not. At
>>>> least I
>>>> >> don't have preconceived ideas :].
>>>> >>
>>>> >>
>>>> >> > Whatever you choose to try, let's keep significant changes for 6
>>>> only :)
>>>> >> >
>>>> >>
>>>> >> Steve was saying that this whole thing wouldn't be an issue in 6.
>>>> >>
>>>> >> --
>>>> >> Guillaume
>>>> >> _______________________________________________
>>>> >> hibernate-dev mailing list
>>>> >> hibernate-dev at lists.jboss.org
>>>> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>>> >
>>>> >
>>>> _______________________________________________
>>>> hibernate-dev mailing list
>>>> hibernate-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>>>
>>>
>>>
>>
>