[hibernate-dev] Memory consumption

[Sanne Grinovero]

Hi Andrej,

Hibernate Search at version 3.0.1 did a small fraction of what it does
today so I would expect indeed it to take a bit more memory but no I
wasn't aware of significant increases.

Looking just at the Search-related figures however it looks like
memory consumption is significantly *lower* than the old times? That's
not really expected, so I'm assuming that some of the memory used by
SessionFactoryImpl, Configuration, JavaReflectionManager is actually
holding to references which are Search related.

In short, it would help us a lot if you could open the details and
explore a bit around the references to understand where things might
have done out of control :-(

Among other improvements in Search, it does now keep open references
to both IndexReaders and IndexWriters, and both consume quite some
memory as they aggressively cache metadata from the indexes. This is a
good thing, as otherwise you would be using this same memory anyway,
but per-hit causing a lot of work for GC but still showing better
figures for the older version as the memory is not really "reserved"..
could you compare them as well by disabling such features in Search ?
Same for FieldCaches and other stuff..

in practice:
 - hibernate.search.​default.​exclusive_index_use = false
 - hibernate.search.default.reader.strategy = not-shared
 - @CacheFromIndex <- all explicitly disabled since it defaults to "class"
 - avoid filter caching

I guess you're comparing them without any application or configuration
changes, still you would need to make these configuration changes to
make the comparison fair as these features didn't exist at the time
and are now enabled by default.
This will disable some benefits you get from latest Search, but not
all of them.. just curious to see if it gets us into more reasonable
limits.

Regards,
Sanne

On 9 May 2012 22:11, Scott Marlow <smarlow at redhat.com> wrote:
> This sounds related to the discussion here
> https://community.jboss.org/message/729224#729224.
>
> How many persistence units do you have defined in the application?  Can
> you get a list of files in the deployment (e.g. run "jar tf
> Appname.?ar") and show us the files.
>
> On 05/09/2012 04:58 PM, Andrej Golovnin wrote:
>> Hi,
>>
>> it seems that screen shots were stripped by the mailing list software.
>> I have uploaded them to flickr. Here are the links:
>>
>> http://www.flickr.com/photos/78207256@N07/7166944408/in/photostream
>> http://www.flickr.com/photos/78207256@N07/7166945108/in/photostream
>>
>> Best regards
>> Andrej Golovnin
>>
>> On 09.05.2012, at 22:35, Andrej Golovnin wrote:
>>
>>> Hi all,
>>>
>>> I have today deployed our application for the first time
>>> on JBoss 7 with the latest Hibernate and Hibernate Search versions.
>>> And I was totally shocked when I saw the memory consumption.
>>>
>>> I have created two screen shots from the memory dumps. The screen shots
>>> are attached to this mail. Please take a look at them. The screen shot
>>> names contains version numbers of the used Hibernate and Hibernate Search
>>> frameworks. The memory dumps were created for one and the same
>>> application version. I can't believe that the latest versions require nearly
>>> twice as much memory as the old versions to perform the same tasks.
>>> What's happen?
>>>
>>> Is this a known problem? Do you have any plans to change this situation,
>>> e.g. to work on reducing memory consumption? Or maybe you already
>>> started to work on this problem?
>>>
>>> We have been running our application with 768MB for max heap size and
>>> serving hundreds of users. And now using the same memory settings I can
>>> just deploy the application. :-(
>>>
>>> Best regards
>>> Andrej Golovnin
>>>
>>> _______________________________________________
>>> hibernate-dev mailing list
>>> hibernate-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>
>>
>> _______________________________________________
>> hibernate-dev mailing list
>> hibernate-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev