[hibernate-dev] Reducing the memory usage of HQLQueryPlan

[Guillaume Smet]

If we can avoid keeping the AST, that would have my preference too.

In 5.x, the AST contains elements that are collected while parsing and then
kept there. We are then forced to keep the AST around. They should be
pushed in a specific container so that we can keep only the strictly
necessary information and get rid of the AST.

6 improvements look great but I agree with Christian that if we can get
entirely rid of the AST, that would be better. Unless there are good
reasons to keep it around.

-- 
Guillaume

On Fri, Aug 10, 2018 at 3:12 PM Christian Beikov <christian.beikov at gmail.com>
wrote:

> I'd like to note that with the array rendering strategy i.e. `where x =
> ANY(?)` and the padding strategy i.e. pad the JDBC params, we can reduce
> the cache trashing and avoid re-walking of the SQL-AST. IMO it's not
> necessary to keep the SQL-AST around for the purpose of parameter
> expansion, but maybe there are other reasons to keep it around?
>
> Wdyt?
>
>
> Mit freundlichen Grüßen,
> ------------------------------------------------------------------------
> *Christian Beikov*
> Am 09.08.2018 um 16:57 schrieb Steve Ebersole:
> > In 6.0 HQLQueryPlan is replaced by AggregatedSelectQueryPlanImpl[1]
> > (polymorphic queries) and ConcreteSqmSelectQueryPlan[2].
> >
> > ConcreteSqmSelectQueryPlan holds a reference to the SQM AST;
> > AggregatedSelectQueryPlanImpl
> > hold 2+ ConcreteSqmSelectQueryPlans.  AggregatedSelectQueryPlan operates
> > much like HQLQueryPlan when holding more than one QueryTranslator.
> >
> > ConcreteSqmSelectQueryPlan ought to hold much less state (and hence use
> > less memory) than QueryTranslator.  And it follows that
> > AggregatedSelectQueryPlan
> > ought to consume less memory than HQLQueryPlan+QueryTranslators.
> >
> > The other win, which is likely much bigger gain, is that previously
> > parameter expansions (think `... where x in (:someListOfValues)`)
> resulted
> > in separate plans based on the number of values in `someListOfValues` -
> > more HQLQueryPlan instances for the same HQL.  This is actually one of
> the
> > (many) explicit design goals for 6.  The cached plan is the same
> regardless
> > of the number of values in `someListOfValues`; the expansion happens
> during
> > execution.
> >
> > Relatedly, this last part also lets us reuse cached plans in more
> scenarios
> > than we do currently: enabled Filters, enabled FetchProfiles, etc.  All
> of
> > these things are applied/resolved during execution.  So there is a slight
> > trade off between reducing the number of cached plans versus translating
> > the SQM AST to a SQL AST.  This is important too because often times the
> > query plan cache is "overrun" due to all of the "different cache keys"
> > aspect (parameter list expansion, filters, etc) - so that often the
> queries
> > get pushed from the cache and need complete re-building.
> >
> > We cannot really verify this though until Luis, et.al, are able to
> resume
> > the performance testing...
> >
> > [1]
> >
> https://github.com/sebersole/hibernate-core/blob/wip/6.0/hibernate-core/src/main/java/org/hibernate/query/sqm/internal/AggregatedSelectQueryPlanImpl.java
> > [2]
> >
> https://github.com/sebersole/hibernate-core/blob/wip/6.0/hibernate-core/src/main/java/org/hibernate/query/sqm/internal/ConcreteSqmSelectQueryPlan.java
> >
> >
> > On Thu, Aug 9, 2018 at 9:21 AM Guillaume Smet <guillaume.smet at gmail.com>
> > wrote:
> >
> >> Hi,
> >>
> >> >From what I can see, the HQLQueryPlan objects are rather big, mostly
> due
> >> to
> >> the sqlAst element of the QueryTranslators.
> >>
> >> They can consume a fair amount of memory when you have a lot of HQL
> >> queries.
> >>
> >> We need at least some of the information collected by the AST but I'm
> >> wondering if we really need to keep all the AST information in memory
> once
> >> the query has been parsed and the SQL query generated.
> >>
> >> The purpose of this email is mostly to know if this element has been
> >> accounted for in 6 development?
> >>
> >> I haven't checked if it would be doable to improve the situation without
> >> breaking too much stuff in 5.x.
> >>
> >> Thanks.
> >>
> >> --
> >> Guillaume
> >> _______________________________________________
> >> hibernate-dev mailing list
> >> hibernate-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >>
> > _______________________________________________
> > hibernate-dev mailing list
> > hibernate-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev