My two cents... Sanne Grinovero, you may want to react to this.

I'm not sure it's a good idea. I'm not saying the "alias" feature in Elasticsearch is useless, but in the case of Hibernate Search, I don't think aliases are enough.

Why it's not a good idea

Main problem: you need to coordinate upgrades

The main reason is, you have to upgrade more than just the index. There's a database, your frontend, your backend, etc., and if you really want zero downtime, you will have to coordinate the upgrades in just the right way. Each component depends on another, and you cannot always make all components work with both the older and newer version of their dependencies.

For instance, let's imagine you do your reindexing, and you want zero downtime. You will probably have application nodes working on the previous version of the index while doing the reindexing. Those nodes will expect the index to be structured in a certain way. If your reindexing process is fully hidden inside Hibernate Search, and if we use aliases, the older index will get replaced by the newer one at some point (you don't know when exactly). When this happens, your "old" application nodes will start failing: the structure of the index changed suddenly, and they received a string result where they expected an integer!

You could say "Hibernate Search should handle multiple mappings simultaneously, we'll just upgrade all our nodes first and then do the reindexing". but that's quite complex, including for Hibernate Search users (see "Working with two mappings simultaneously" below).

So you need, somehow, to be able to take down the non-upgraded application nodes before the aliases are changed.

And then you start wondering what sort of benefit hiding aliases inside mass indexing would bring to you, since you need to execute operations between the actual reindexing and the alias change...

Secondary problem: document updates during indexing should not be lost

There is another reason aliases are not enough: during reindexing, you don't want document updates to be lost. You want Hibernate Search to continue sending updates to the index according to your database updates, just in case an entity is updated between when it has been reindexed and when we switch from the old index to the new one. You could simply keep sending those updates to the old version of the index, but then those updates will ultimately be lost when switching to the new version of the index.

So you will expect Hibernate Search to somehow send the updates to the new index. But then you will have to provide the name of this new index beforehand: Hibernate Search may not be aware of it, since the reindexing may be happening on another application node.

And then you start wondering what sort of benefit hiding aliases would bring to you, since you need to provide the "low-level" name of the targeted index to Hibernate Search anyway...

Solution?

The only "simple" way I know to avoid the issues is to orchestrate the index on a per-node basis:

• keep at least one non-upgraded application node online to handle read operations
• spawn upgraded application nodes for write operations and reindexing, not exposed to the outside world, targeting a new, unaliased index (e.g. my_index_v2 instead of the alias my_index).
• switch the non-upgraded nodes to read-only mode and redirect all write operations to the upgraded nodes
• start reindexing
• once reindexing is complete, take the upgraded nodes fully online and take the non-upgraded application nodes offline (to upgrade and restart them)
• and only then switch back the upgraded nodes from targeting the low-level my_index_v2 name to simply targeting the alias, my_index.

But then, using aliases is just cosmetics. You can see that the last step is not strictly necessary, everything would still work without it. Also, they introduce new fancy ways to shoot yourself in the foot, like switching back to aliased names for only half of the indexes.

Maybe we could simply provide an configuration option to add a suffix to every index, making it easy for you to change it from _v1 to _v2 when you do zero-downtime upgrades. But that's about the only thing users would need, in my opinion.

Implicit features

Your ticket implicitly requests the following features:

1. Reindexing on startup
2. Working with two mappings simulatenously (the new one, and the one currently in the index)

Below are some details as to why they would be problematic.

Reindexing on startup

Reindexing may take hours, and in most cases it will take a few minutes at least. It's just way too long, especially considering that most applications will wait for Hibernate ORM (and thus Search) to be initialized before starting other components on the server, such as the REST services or the web application framework. Such a long startup would lead to timeouts, or to web applications that are unavailable until reindexing is done, which defeats the whole purpose. As explained above, it's probably better if Hibernate Search users can orchestrate the upgrade on their own, after Hibernate Search started.

Working with two mappings simulatenously

You scenario implicitly requires Hibernate Search to deal with both the old mapping (before upgrade) and the new mapping (after upgrade). This is much more complex that it seems.

First, we would need metadata about the old mapping. Elasticsearch metadata is not always enough to allow Hibernate Search to perform queries, we need the mapping metadata too (in particular to know which bridges are used). The old mapping metadata is obviously not available once you changed your mapping (removed @Field annotations, changed some other, ...), so the Hibernate Search user would have to provide it explicitly, somehow.

Second, you may have changed your entity model (added/changed/removed properties from your entities), so we wouln't need the old mapping exactly, but a mapping from your new entity model to your old index model. It would require quite some work from you, as a Hibernate Search user.