[hibernate-dev] Reducing the memory usage of HQLQueryPlan

[Steve Ebersole]

In 6.0 HQLQueryPlan is replaced by AggregatedSelectQueryPlanImpl[1]
(polymorphic queries) and ConcreteSqmSelectQueryPlan[2].

ConcreteSqmSelectQueryPlan holds a reference to the SQM AST;
AggregatedSelectQueryPlanImpl
hold 2+ ConcreteSqmSelectQueryPlans.  AggregatedSelectQueryPlan operates
much like HQLQueryPlan when holding more than one QueryTranslator.

ConcreteSqmSelectQueryPlan ought to hold much less state (and hence use
less memory) than QueryTranslator.  And it follows that
AggregatedSelectQueryPlan
ought to consume less memory than HQLQueryPlan+QueryTranslators.

The other win, which is likely much bigger gain, is that previously
parameter expansions (think `... where x in (:someListOfValues)`) resulted
in separate plans based on the number of values in `someListOfValues` -
more HQLQueryPlan instances for the same HQL.  This is actually one of the
(many) explicit design goals for 6.  The cached plan is the same regardless
of the number of values in `someListOfValues`; the expansion happens during
execution.

Relatedly, this last part also lets us reuse cached plans in more scenarios
than we do currently: enabled Filters, enabled FetchProfiles, etc.  All of
these things are applied/resolved during execution.  So there is a slight
trade off between reducing the number of cached plans versus translating
the SQM AST to a SQL AST.  This is important too because often times the
query plan cache is "overrun" due to all of the "different cache keys"
aspect (parameter list expansion, filters, etc) - so that often the queries
get pushed from the cache and need complete re-building.

We cannot really verify this though until Luis, et.al, are able to resume
the performance testing...

[1]
https://github.com/sebersole/hibernate-core/blob/wip/6.0/hibernate-core/src/main/java/org/hibernate/query/sqm/internal/AggregatedSelectQueryPlanImpl.java
[2]
https://github.com/sebersole/hibernate-core/blob/wip/6.0/hibernate-core/src/main/java/org/hibernate/query/sqm/internal/ConcreteSqmSelectQueryPlan.java


On Thu, Aug 9, 2018 at 9:21 AM Guillaume Smet <guillaume.smet at gmail.com>
wrote:

> Hi,
>
> >From what I can see, the HQLQueryPlan objects are rather big, mostly due
> to
> the sqlAst element of the QueryTranslators.
>
> They can consume a fair amount of memory when you have a lot of HQL
> queries.
>
> We need at least some of the information collected by the AST but I'm
> wondering if we really need to keep all the AST information in memory once
> the query has been parsed and the SQL query generated.
>
> The purpose of this email is mostly to know if this element has been
> accounted for in 6 development?
>
> I haven't checked if it would be doable to improve the situation without
> breaking too much stuff in 5.x.
>
> Thanks.
>
> --
> Guillaume
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>