[hibernate-issues] [JIRA] (HSEARCH-3900) drop-and-create strategy is not working at app. startup when "hibernate.search.autoregister_listeners" is set to false. And if true, startup index creation thread are not released, sending the app in out of memory.

[Ronak S (JIRA)]

Ronak S ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=5ea56c8496bbcb0b852580f0 ) *commented* on HSEARCH-3900 ( https://hibernate.atlassian.net/browse/HSEARCH-3900?atlOrigin=eyJpIjoiMmY4NTYxZjQ2Njk3NDFiYTkwNDM2YTQ5YzAzNTNjMTIiLCJwIjoiaiJ9 )

Re: drop-and-create strategy is not working at app. startup when "hibernate.search.autoregister_listeners" is set to false. And if true, startup index creation thread are not released, sending the app in out of memory. ( https://hibernate.atlassian.net/browse/HSEARCH-3900?atlOrigin=eyJpIjoiMmY4NTYxZjQ2Njk3NDFiYTkwNDM2YTQ5YzAzNTNjMTIiLCJwIjoiaiJ9 )

Yoann, thanks for the response.

I’ve few queries with reference to your answer. Below are the same:

> 
> 
> 
> Regardless... I'd stay away from this setting. It probably doesn't do what
> you think it does.
> 
> 

We have been using Hibernate Search from last 1.5 years on below version and these are newer than 5.5.

We still never faced this issue, we even did not manually provided the configuration of autoregister_listeners in our hibernate.cfg file. Still it was all well, the problem only started after upgrading to 5.11.5.

> 
> 
> 
> If you want to enable Hibernate Search, but prevent automatic indexing,
> see here (
> https://docs.jboss.org/hibernate/search/5.11/reference/en-US/html_single/#_automatic_indexing
> ) instead. You just have to set this:
> 
> 
> 
> hibernate.search.indexing_strategy = manual
> 
> 
> 
> Hibernate Search will still start up, but entity persisting events will be
> ignored. You will have to use the mass indexer in order to index your
> data, or explicit index commands. See here (
> https://docs.jboss.org/hibernate/search/5.11/reference/en-US/html_single/#search-batchindex
> ) for more information.
> 
> 

I can not set it to manual, as in our application we have used hibernate search to update the elastic database as an when we have an add/update/delete on the entities which are @Indexed annotated. For eg. If a user adds a client data, this needs to be added on elastic server, as the search part is provided through elastic server.

> 
> 
> 
> I'll need to know which setting you're talking about. Both autoregister_listeners
> and indexing_strategy were introduced before version 5.6 , so long before
> Elasticsearch support was event introduced.
> 
> 

I’m talking about autoregister_listeners and indexing_strategy settings. As now if I keep the autoregister_listeners to false then my drop-and-create strategy does not work any longer. As we require this to update the schema on elastic side, once we modify any entity to include/exclude fields from elastic schema. These were even working correctly under 5.9 version.

> 
> 
> 
> My concern is, we do have an option of create-and-drop of indexes at sever
> startup, are we going to remove this option?
> 
> 
> 
> Which you replied with “No“
> 
> 

If we are not planning to remove this and it will be available. How do you suggest of doing this with current scenario. Where if autoregister_listeners are true than we have the performance issue and when it is false, the whole hibernate search is disabled in the application?

> 
> 
> 
> No. It just removes all documents, but it preserves the same schema.
> 
> 
> 
> The option to drop and recreate indexes in the mass indexer was introduced
> in Search 6.0.0.Beta6 ( HSEARCH-3751 (
> https://hibernate.atlassian.net/browse/HSEARCH-3751 ) ); see the dropAndCreateSchemaOnStart
> parameter here (
> https://docs.jboss.org/hibernate/search/6.0/reference/en-US/html_single/#mapper-orm-indexing-massindexer-parameters
> ).
> 
> 

See, we can only update the data with startAndWait, but how will I refresh my schema with this as that is only possible with drop-and-create, ryt?

Further, we were planning to go to the latest version, but then it’s in BETA stage. So we can not adapt a BETA version for our product. Do you suggest to move to the version 6 BETA, will doing this resolve all these issues?

> 
> 
> 
> No, that's useful to trigger mass indexing, reindex your data while taking
> your application offline. After that, you can restart your application and
> re-enable automatic indexing.
> 
> 
> 
> Some people also don't use automatic indexing at all and just reindex
> their whole database periodically. That's not for everyone, though.
> 
> 

MassIndexing is fine and we do it the way you are suggesting with startAndWait(), but how do I achieve the refresh of schema. Also as you said its not the case with everyone, and yes we can not do periodic refresh, instead we need it at runtime. As I explained above in one of the comments.

> 
> 
> 
> As I already explained, setting hibernate.search.autoregister_listeners to
> false will just *disable Hibernate Search*. So yes, that's expected.
> 
> 

Are you suggesting that now the automatic indexing on add/update/delete support is lost from 5.11. Because honestly it was all fine in 5.9.

> 
> 
> 
> Where do the problems seem to happen? Just when booting? Or when the
> application handles HTTP requests that result in write operations on the
> database?
> 
> 

It’s only at the server booting/startup time. Otherwise the application is working as expected.

> 
> 
> 
> Did you change anything at all?
> 
> 
> 
> Changes in your model and Search mapping could lead to dramatic
> performance problems if they trigger a lot of reindexing ( @ContainedIn ).
> I'm talking about indexing one entity leading to loading half the database.
> You need to be very careful about what you do with @ContainedIn.
> 
> 
> 
> Switching from Lucene to Elasticsearch can also represent a high cost if
> your application is very write-intensive. In particular, you should not
> enable hibernate.search.default.elasticsearch.refresh_after_write. This
> setting will lead to terrible performance in write-intensive applications.
> 
> 
> 

No I did not make any change with respect to @ContainedIn we had this functionality in 5.9 too. And yes we do have @ContainedIn relationships in elastic.

Further, we were already on lucene and did not migrate to Elasticsearch. Also I’ve not provided hibernate.search.default.elasticsearch.refresh_after_write in my hibernate config.

At last,

> 
> 
> 
> As I explained above, with autoregister_listeners set to false , Hibernate
> Search is disabled. So of course all Hibernate Search features are
> disabled as well.
> 
> 

If this setting is off, I’m left with just the hibernate search jars in my application, doing nothing 🙂.

Please help me to overcome this issue, as now I’m more curious to understand why it was fine in 5.9 and not now with 5.11?

Just look at this scenario in a way that my application is now dead from hibernate search side.

Please revert if you need more details on configuration, jar dependencies we have or any other info you require to understand it in more better way.

( https://hibernate.atlassian.net/browse/HSEARCH-3900#add-comment?atlOrigin=eyJpIjoiMmY4NTYxZjQ2Njk3NDFiYTkwNDM2YTQ5YzAzNTNjMTIiLCJwIjoiaiJ9 ) Add Comment ( https://hibernate.atlassian.net/browse/HSEARCH-3900#add-comment?atlOrigin=eyJpIjoiMmY4NTYxZjQ2Njk3NDFiYTkwNDM2YTQ5YzAzNTNjMTIiLCJwIjoiaiJ9 )

Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.core&referrer=utm_source%3DNotificationLink%26utm_medium%3DEmail ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=EmailNotificationLink&mt=8 ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100125- sha1:c966059 )
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/hibernate-issues/attachments/20200427/7e0ef3ed/attachment.html