Quoting:

[2:46 PM] Yoann Rodière: @Sanne I may be wrong, but I think there's a problem with how dynamic sharding works... (yes, another )
IIUC, one is supposed to override ShardIdentifierProviderTemplate.loadInitialShardNames(Properties, BuildContext) so that index managers are correctly restored on startup.
The obvious place to get those initial shard identifiers from would be the database. But from what I can see, users cannot access their database from here (except using some Hibernate Search internals such as org.hibernate.search.hcore.impl.HibernateSessionFactoryService). So basically they can use dynamic sharding, but cannot reliably query the whole index (each query must have a shard filter for it to work)...
[2:47 PM] Sanne Grinovero: @yoann yes I'm aware, there have been user requests to have access to the Session... anyone working on injection capabilities ?

[2:54 PM] Yoann Rodière: In the long run we may want to store this list of identifier ourselves... I suspect we could avoid it with Elasticsearch anyway, if we use _route instead of actually creating separate indexes. For Lucene, well, we already store stuff on the disk, so what's a single little file, and for JGroups/JMS well we're already screwed anyway
[2:55 PM] Sanne Grinovero: @yoann yes, if the user isn't giving us the list in advance we can always build it up (and keep) ourselves

We discussed two solutions for now:

• Allow dependency injection in ShardIdentifierProvider, so that users can retrieve shard identifiers from the database. This would be solved by HSEARCH-2658 Open .
• Store the list of identifiers ourselves. This would require HSEARCH-2674 Open , some changes in Elasticserach to use _route for dynamic sharding, plus some more work.