actually, thinking about it more, not so sure the "reusing keys"
approach would not work.
On Wed 16 May 2012 04:35:36 PM CDT, Steve Ebersole wrote:
Well another option is to change the way batch load sql is
created/handled which is something that has come up many times before,
and someone was just discussing on the list last few weeks.
Given a batch size, Hibernate currently generates a non-unintuitive
number of loaders. Lets look at an example number. You gave 16 as the
batch size. That means Hibernate would generate loaders for sizes:
16,10,9,8,7,6,5,4,3,2,1
The idea being how to efficiently load batches when the number of
things to load is less that 16.
Deferring initialization of these batch loaders is one option, but
that just really delays the memory consumption. Personally, if it were
my app I'd rather that sizing be done up front rather than over the
course of the next few hours or weeks as the system continues to run.
Its been proposed a number of times to instead generate just a single
loader for loading that number of items. This would mean generating a
single SQL statement that has parameter holders for the full batch
size and then somehow "filling out" the empty slots when the number of
things to load is less that 16. If this could be made workable across
all dialects, I personally think it would be the best approach as it
probably gets close to the start up sizings you claimed but would
never grow beyond that. But we would need to discuss this to see if it
is a real possibility. Lets work an example with batch-size = 3. That
would produce SQL like "select a.x, a.y, a.z from a a where a.x = ? or
a.x = ? or a.x = ?"
So for a case where we have 2 things to batch load, how do we best
leverage that single statement? Which really comes down to how we
handle the the 3rd parameter binding.
I have seen simply reusing one of the keys as a proposal, but that
will not work. Null should work, but would it work in all cases
against all dialects?
On 05/16/2012 03:14 PM, Andrej Golovnin wrote:
> Hi Scott,
>
>>
>> Nice find! Could you drill into one of the LockMode loaders and see
>> what the larger objects referenced by that are?
>>
>
> Largest objects are of type BatchingEntityLoader.
>
> The problem has to do with the value of
> hibernate.default_batch_fetch_size.
> In our case we use 16 as the value. In this case Hibernate creates 11
> EntityLoaders
> for every entity class involved in a collection association. So the
> most memory
> is consumed by generated SQL statements. If I compare memory snapshots,
> I see ca. 100MB more of String objects in JBoss 7 as in JBoss 4.
>
> Changing the value of hibernate.default_batch_fetch_size from 16 to 4
> reduces the size of SessionFactoryImpl in our case from ca. 370MB to
> 180MB.
> And if I use patched version of Hibernate as suggested by me, the
> size of
> SessionFactoryImpl is ca. 80MB. So we will now investigate, wether it is
> really needed to set hibernate.default_batch_fetch_size to 16 or maybe
> the lower value would also give us decent performance.
>
> Best regards
> Andrej Golovnin
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/hibernate-dev