[hibernate-dev] Hibernate Search and Elasticsearch 5 support

[Yoann Rodiere]

Hello,

I just finished assessing the required changes for supporting Elasticsearch
5.0. I put the details in this ticket:
https://hibernate.atlassian.net/browse/HSEARCH-2434

Here is a quick summary:

   - Some (non-breaking) changes are required in Jest [1]
   - We'll have to update the way we do integration testing:
   elasticsearch-maven-plugin doesn't work well with ES 5.0 and will require a
   major overhaul to work. [2]
   - There are some breaking changes that require us either drop support
   for ES 2.x or introduce dialects (for instance the string datatype has been
   split into two datatypes: text and keyword, which behave quite differently).
   - And, perhaps most importantly, support for defining analyzers in
   elasticsearch.yml has been dropped. This means users have to resort to the
   index settings API to define their analyzers. So this breaks our automatic
   index creation / mapping generation feature: we put mappings just after
   creating the index, but since the index was just created analyzer
   definitions will be missing and the mapping will be rejected. See
   HSEARCH-2434 (in the comments) for details.

Ultimately, what we have to decide is how and when we're going to support
ES 5. Several options have already been mentioned on HipChat:

   1. Support ES 5.x right now and drop support for ES 2.x
   2. Support both ES 5.x and 2.x right now by introducing dialects (that
   could be chosen automatically by asking the running version to the server)
   3. Only support ES 2.x for now and keep ES 5.x for later (probably HS
   6.0)

We have to consider several things in order to make the decision:

   - Deadlines and available resources. Supporting 5.x only should be easy
   enough, I think (if we ignore the analyzer definition issue) ; actually I
   already did some work [3]. The dialect solution would obviously require
   some more work, but if only target a quick, dirty fix (to be refactored in
   6.0) it shouldn't be hell.
   - Users. Dropping support for 2.x probably will make someone angry. But
   then, we only published alpha/betas and advertised experimental support for
   now, so there's shouldn't be many people using it in a production
   environment.


Thoughts, opinions?


[1] https://github.com/searchbox-io/Jest/pull/408
[2] https://github.com/alexcojocaru/elasticsearch-maven-plugin/pull/19
[3] https://github.com/yrodiere/hibernate-search/tree/HSEARCH-2434


Yoann Rodière <yoann at hibernate.org>
Hibernate NoORM Team