[hibernate-issues] [JIRA] (HSEARCH-3900) drop-and-create strategy is not working at app. startup when "hibernate.search.autoregister_listeners" is set to false. And if true, startup index creation thread are not released, sending the app in out of memory.

[Yoann Rodière (JIRA)]

Yoann Rodière ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3A58fa1ced-171a-4c00-97e8-5d70d442cc4b ) *commented* on HSEARCH-3900 ( https://hibernate.atlassian.net/browse/HSEARCH-3900?atlOrigin=eyJpIjoiY2VjZjMzNzJiNDhmNDFiYjgwZDNjMTI4OGU2OTVlYTAiLCJwIjoiaiJ9 )

Re: drop-and-create strategy is not working at app. startup when "hibernate.search.autoregister_listeners" is set to false. And if true, startup index creation thread are not released, sending the app in out of memory. ( https://hibernate.atlassian.net/browse/HSEARCH-3900?atlOrigin=eyJpIjoiY2VjZjMzNzJiNDhmNDFiYjgwZDNjMTI4OGU2OTVlYTAiLCJwIjoiaiJ9 )

> 
> 
> 
> We have this set correctly.
> 
> 

Then you probably have something like that somewhere:

hibernate.search.myIndex1.indexmanager = <something that is not "elasticsearch" >

Specifically, I can tell you from your thread dump that "TABLEWC30" is using a Lucene index, not an Elasticsearch one.

> 
> 
> 
> But on startup we are not indexing anything.
> 
> 

The transport threads are also used to send requests to Elasticsearch to drop/create indexes, or simply to execute search queries. My point is, if they do something, it's because you asked them to.

> 
> 
> 
> > The threads will not stop as long as your application is running. That's
> on purpose.
> 
> 

> 
> 
> 
> Why is so?
> 
> 

Because spawning new threads every time we need to do something would be disastrous performance-wise.

> 
> 
> 
> > Was your only problem that "Hibernate Search creates threads"?
> 
> 

> 
> 
> 
> No, issue is these threads are throwing my application in out of memory
> condition.
> 
> 

Let's say you have two threads T1 and T2, and 500MB of memory available. Let's say T1 allocates 499MB of memory, then T2 allocates 2MB of memory. The OutOfMemoryError will occur in T2, but obviously T1 is the biggest consumer of memory.

Just because the OutOfMemoryError happpened in these threads, doesn't necessarily mean these threads are the source of your trouble. They could be, but that's far from certain.

> 
> 
> 
> Also If I switch elastic search off in my app, then the application runs
> fine.
> 
> 

Certainly. If you remove Hibernate Search *and* Elasticsearch from your application, you will need less memory. That doesn't mean that Hibernate Search and Elasticsearch consume too much memory; maybe there is simply not enough memory available.

In your case, the Elasticsearch transport threads are probably just serializing JSON to sent requests to drop/create indexes. Unless you have a very, very complicated schema, I wouldn't expect that to cause too many memory allocations. On the Elasticsearch side, however, the creation of an index can result in significant memory allocations. Maybe Elasticsearch is eating up the memory and your application no longer has enough RAM available?

What is the amount of RAM available on your server, and what are the memory flags ( -Xms / -Xmx on your JVMs (Tomcat *and* Elasticsearch)? What are the processes using the most memory when you encounter your OutOfMemoryError , and how much memory are they using?

Also, if you actually have data that points toward Hibernate Search allocating too much memory (for example Java Flight Recorder recordings), please show me that data, so I can pinpoint which class allocates so much. You can find various tutorials on the web, [this one] ( https://stackoverflow.com/a/54143988/6692043 ) looks like a good starting point.

( https://hibernate.atlassian.net/browse/HSEARCH-3900#add-comment?atlOrigin=eyJpIjoiY2VjZjMzNzJiNDhmNDFiYjgwZDNjMTI4OGU2OTVlYTAiLCJwIjoiaiJ9 ) Add Comment ( https://hibernate.atlassian.net/browse/HSEARCH-3900#add-comment?atlOrigin=eyJpIjoiY2VjZjMzNzJiNDhmNDFiYjgwZDNjMTI4OGU2OTVlYTAiLCJwIjoiaiJ9 )

Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.core&referrer=utm_source%3DNotificationLink%26utm_medium%3DEmail ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=EmailNotificationLink&mt=8 ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100125- sha1:29f7b81 )
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hibernate-issues/attachments/20200428/b83c0945/attachment.html