[hibernate-dev] Hibernate Search connecting to multiple Elasticsearch clusters

[Sanne Grinovero]

On 8 March 2016 at 12:04, Gunnar Morling <gunnar at hibernate.org> wrote:
> Hi,
>
> Good question, I have been wondering the same.
>
> I think we could even support this 5.6, essentially it's just a matter
> of structuring the properties and how we read them. I basically
> started with a single cluster to get to something working more
> quickly, but it shouldn't take long to change that now.
>
> Only in the docs we need to make clear that index sharing / queries do
> not work across cluster boundaries.
>
> If we don't do it now, at least let's change the prop name into
> "hibernate.search.default.elasticsearch.host" so it's not breaking if
> we add support for index-specific overrides down the road.


Yes it sounds like we agree about HSEARCH-2164 being a blocker.

But will we regret this? what if we later decide that supporting
multiple Elasticsearch clusters is both of low value and too
problematic?
Not sure if we're yet in a position to be aware of all implications; I
guess I shouldn't worry too much as this is still experimental, yet I
think it would be good to go through our list of features via the
reference documentation and try thinking ahead of what problems that
will introduce.

I didn't find any point which wouldn't be easy to dismiss but a second
pair of brains would be nice on this ;)

Thanks,
Sanne



>
>
> --Gunnar
>
> 2016-03-08 12:55 GMT+01:00 Sanne Grinovero <sanne at hibernate.org>:
>> Today our experiments are assuming we're connecting to a single
>> Elasticsearch cluster: one hostname to configure, etc..
>>
>> I think this is an acceptable limitation for version 5.6 (our first
>> stable milestone to support this integration) but I'm wondering if we
>> should document it as a temporary limitation or as an intentional
>> design.
>>
>> I think that eventually we should allow having different entities
>> (indexes) to be stored on different ES clusters; this shouldn't be too
>> hard to manage as the codebase is already structured around this
>> capability of having different types in different "indexes"; so while
>> a single Elasticsearch cluster can manage multiple indexes (that's a
>> bit of a novel concept) I see no reason to not allow different indexes
>> to be mapped on different clusters.
>>
>> = Am I missing some strong reason to not allow this?
>>
>> = While this will (likely) not be supported in 5.6, should the public
>> API and configuration properties allow for this to be configured
>> per-index already? (Changing it later would be a breaking change)
>>
>> See also:
>>  - https://hibernate.atlassian.net/browse/HSEARCH-2164
>>
>> Thanks,
>> Sanne
>> _______________________________________________
>> hibernate-dev mailing list
>> hibernate-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev