Hello Yoann,

1. I have multi-tenant data and it is stored in a single database, and I use a discriminating property to differentiate between tenants. However, I do not use Hibernate's multi-tenancy strategy, and instead, I manage multi-tenant security on my own. More specifically, I apply a filter (@Filter and @FilterDef) to the Hibernate session that applies the tenant discriminator, whenever I need to target a specific tenant. This allows me to dynamically enable / disable multi-tenancy from queries. I also handle multi-tenancy myself when using Hibernate Search.

2. The reason was technical - I was concerned about scalability of my cluster (which has 4 nodes by the way) - in terms of how many replicas I could add. But now I re-evaluated my decision and it turns out there is no difference in scalability if I use 3 primaries or 6 primaries.

There is still the issue of resource cost per shard - the more shards there are in a node, the more CPU, memory and file handles are going to be consumed and there is a risk of contention (see this article). But maybe I am over-optimizing. I am OK with merging the 2 LIVE indexes into one, I will try it out.

3. Dynamic sharding, as it works right now, allows me to select the proper ES index when indexing a document (ARCHIVE or LIVE), which is good. I need this. However, since I cannot select additionally the ES shard (internally a separate Lucene index) to put the document in, it means documents for a specific organisation will be spread around many shards in the ES index. I have thousands of organisations, and creating thousands of ES indexes (one per org) is not an option. If you adapt dynamic sharding to map to ES shards instead of ES indexes, that will not fit my use case, because I still need to put some documents in ARCHIVE and others - in LIVE.

The main value _routing brings to me is the fact that even though I have thousands of domains (organisations), I can put them in just a few ES shards, thanks to the hash function. In this way all documents of an organisation fit into a single shard, and I still don't have to manage thousands of shards or indexes. My indexing plan will also not break if I scale the cluster.

NOTE:
I am implementing this ES integration in parallel with you, and it is not live yet. I am also searching for a way to extent Hibernate Search 5.6.1.Final to support _routing (even in a hardcoded way) but I cannot find an extension point for that.