[JIRA] (HSEARCH-3499) Near-zero-downtime mass indexing
by Yoann Rodière (JIRA)
Yoann Rodière ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) *commented* on HSEARCH-3499 ( https://hibernate.atlassian.net/browse/HSEARCH-3499?atlOrigin=eyJpIjoiZTF... )
Re: Near-zero-downtime mass indexing ( https://hibernate.atlassian.net/browse/HSEARCH-3499?atlOrigin=eyJpIjoiZTF... )
>
>
>
> Is there a way to access the (expected) mapping of a given index?
>
>
At the moment, no, there isn't a way to access it programmatically. The best you can do is to run the schema creation in a development environment and get the resulting mapping from Elasticsearch.
>
>
>
> But when creating the new index (by calling the ES API directly) I don’t
> see how to post the index mappings along with the new index.
>
>
I think you mean you don't see how to guess the new index mapping that you must create in the new index? But just in case, here ( https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-c... ) is how to post the mapping while creating the index.
>
>
>
> Am I taking the wrong approach or just missing how to access the updated
> IndexMetadata?
>
>
To be honest there is something wrong: zero-downtime reindexing as described in the documentation will only work if your mapping did *not* change. If your mapping did *not* change, you can use the rollover API ( https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-r... ) to create the new index without knowing anything about the mapping.
This ticket, and the methodology described in the documentation, will work for periodic reindexing, but definitely are not enough for application updates. I suggest you have a look at HSEARCH-2861 ( https://hibernate.atlassian.net/browse/HSEARCH-2861 ) Open and its comments, where we discussed a few of the problems involved in zero-downtime application updates. The main problem is that old-gen instances of your applications should be allowed to write anything after you created the new index, because they will generate wrong or incomplete documents. Conversely, new instances of your application may not be able to read from the old index, since they have a different metamodel and may assume fields are present while they are not, or assume fields have a different type than they have in the old index.
So, the only "generic", guaranteed-to-work solution here would be to completely separate your applications:
* Make old-gen applications read-only, and I mean *really* read-only. They must not write *anything* , or your indexes will become out-of-sync at best, or will contain invalid data in the worst case).
* Make new-gen applications write-only. They must not handle search requests, because they don't understand the old mapping anymore.
As to how you will do that... I'll leave the routing of user requests to you, because Hibernate Search simply can't handle that: when it becomes involved in the read or write process, it's already too late. I'll just point out that if you go to such lengths, you may as well assign a different name to your index in the new version of your application ( @Indexed(name = "myindex-v2" , so you will have completely separate aliases and indexes for your two applications. You may be able to take advantage of a custom layout ( https://docs.jboss.org/hibernate/search/6.0/reference/en-US/html_single/#... ) strategy that automatically appends the version of your application to your index name and aliases ( myindex-appV2-000001 , myindex-appV2-read , myindex-appV2-write ), but that's about as far as Hibernate Search can help you.
( https://hibernate.atlassian.net/browse/HSEARCH-3499#add-comment?atlOrigin... ) Add Comment ( https://hibernate.atlassian.net/browse/HSEARCH-3499#add-comment?atlOrigin... )
Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.... ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=EmailN... ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100129- sha1:98c8491 )
5 years, 9 months
[JIRA] (HHH-14075) Changes to loaders and TwoPhaseLoad to allow "internal" loading to be reused by hibernate-reactive
by Gail Badner (JIRA)
Gail Badner ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) *created* an issue
Hibernate ORM ( https://hibernate.atlassian.net/browse/HHH?atlOrigin=eyJpIjoiZWVlNmYzYzVh... ) / Task ( https://hibernate.atlassian.net/browse/HHH-14075?atlOrigin=eyJpIjoiZWVlNm... ) HHH-14075 ( https://hibernate.atlassian.net/browse/HHH-14075?atlOrigin=eyJpIjoiZWVlNm... ) Changes to loaders and TwoPhaseLoad to allow "internal" loading to be reused by hibernate-reactive ( https://hibernate.atlassian.net/browse/HHH-14075?atlOrigin=eyJpIjoiZWVlNm... )
Issue Type: Task Assignee: Gail Badner ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... ) Components: hibernate-core Created: 17/Jun/2020 16:07 PM Fix Versions: 5.5.0.Beta1, 6.0.0.Alpha6, 5.4.17 Priority: Major Reporter: Gail Badner ( https://hibernate.atlassian.net/secure/ViewProfile.jspa?accountId=557058%... )
Includes the following changes.
* Break TwoPhaseLoad into 2 methods, #initializeEntityEntryLoadedState and #initializeEntityFromEntityEntryLoadedState. #initializeEntityEntryLoadedState takes an argument of type EntityResolver , which is responsible for resolving to-one entity properties.
* A method was extracted from Loader#processResultSe , #getRowsFromResultSet , which reads all the rows from the ResultSet.
* Some methods in Loader and OuterJoinLoader were changed to protected or public , as needed so that Reactive implementations could make them public.
* Changes to AbstractLoadPlanBasedEntityLoader , AbstractLoadQueryDetails , EntityLoadQueryDetails , BatchingLoadQueryDetailsFactory , to allow a ResultSetProcessorResolver to be provided by an implementation of AbstractLoadPlanBasedEntityLoader
* Changes to ResultSetProcessorImpl to allow hibernate-reactive loaders to extend it so that it can process the ResultSet and resolve hydrated objects reactively.
* Added EntityType#isEager(Boolean overridingEager) in order to determine if an entity attribute being resolved by TwoPhaseLoad#initializeEntity will be eager.
( https://hibernate.atlassian.net/browse/HHH-14075#add-comment?atlOrigin=ey... ) Add Comment ( https://hibernate.atlassian.net/browse/HHH-14075#add-comment?atlOrigin=ey... )
Get Jira notifications on your phone! Download the Jira Cloud app for Android ( https://play.google.com/store/apps/details?id=com.atlassian.android.jira.... ) or iOS ( https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=EmailN... ) This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100129- sha1:3ed9692 )
5 years, 9 months