[hibernate-dev] About HSEARCH-310

[Sanne Grinovero]

Hello,
I'm in need of some help about how to fix this memory leak at HSEARCH-310
in ScrollableResultsImpl of Search, there currently is:

* an internal cache of all loaded EntityInfo
* an internal cache of all loaded managed Entities

Both caches are never emptied during the scrolling; the implementation
is loading a batch-sized list each time an entity is requested which
was not loaded before and adds new objects to the caches.

We had said to use some Soft ReferenceMap to fix this, but actually we
should evict the loaded entities which where not requested from the
client code, or I'll still be leaking memory in Hibernate's
persistence context.

I expect the most frequent use case for a ScrollableResults should be
to iterate at will and periodically evict objects in the client code;
Still the client could skip some entities we loaded by batching.
At first I thought to keep track of which references were used and
evict the other ones, but this isn't a good solution as the client
could be loading some entities by other means (i.e. another query) and
won't be happy if I'm evicting some random entities: I don't have
enough information to know for sure what to evict and what is going to
be used later.

Also a bug, consequence of current implementation, is that if you get
an object, then you evict it, and then try scrollable.get() you don't
get an attached object but again the evicted instance; as a client of
the Hibernate library I expect all objects I ask for to be attached by
default.

My idea to fix this should be something like:
 1) load the batch
 2) evict them all
 3) and then load one by one as requested, relying on the second level
cache (I suppose point 1 should have warmed it)

But this has other flaws.. like I should not evict (2) those objects
already existing in the persistence context.
Ideally I should avoid to preload objects already present in the
persistence context; I am considering to look in the
session.persistenceContext.entitiesByKey map to only load+evict by
batch those entities not already managed.

Is there some way to populate the second level cache skipping the
actual load in the persistence context?
Should I keep my own "soft map cache" in the scroll implementation
anyway, and somehow detach and reattach them as needed?
Am I missing some trivial better solution, or is my solution not going
to work at all?

thanks,
Sanne