6:38 a.m.
Hello,
I just finished assessing the required changes for supporting Elasticsearch
5.0. I put the details in this ticket:
https://hibernate.atlassian.net/browse/HSEARCH-2434
Here is a quick summary:
- Some (non-breaking) changes are required in Jest [1]
- We'll have to update the way we do integration testing:
elasticsearch-maven-plugin doesn't work well with ES 5.0 and will require a
major overhaul to work. [2]
- There are some breaking changes that require us either drop support
for ES 2.x or introduce dialects (for instance the string datatype has been
split into two datatypes: text and keyword, which behave quite differently).
- And, perhaps most importantly, support for defining analyzers in
elasticsearch.yml has been dropped. This means users have to resort to the
index settings API to define their analyzers. So this breaks our automatic
index creation / mapping generation feature: we put mappings just after
creating the index, but since the index was just created analyzer
definitions will be missing and the mapping will be rejected. See
HSEARCH-2434 (in the comments) for details.
Ultimately, what we have to decide is how and when we're going to support
ES 5. Several options have already been mentioned on HipChat:
1. Support ES 5.x right now and drop support for ES 2.x
2. Support both ES 5.x and 2.x right now by introducing dialects (that
could be chosen automatically by asking the running version to the server)
3. Only support ES 2.x for now and keep ES 5.x for later (probably HS
6.0)
We have to consider several things in order to make the decision:
- Deadlines and available resources. Supporting 5.x only should be easy
enough, I think (if we ignore the analyzer definition issue) ; actually I
already did some work [3]. The dialect solution would obviously require
some more work, but if only target a quick, dirty fix (to be refactored in
6.0) it shouldn't be hell.
- Users. Dropping support for 2.x probably will make someone angry. But
then, we only published alpha/betas and advertised experimental support for
now, so there's shouldn't be many people using it in a production
environment.
Thoughts, opinions?
[1] https://github.com/searchbox-io/Jest/pull/408
[2] https://github.com/alexcojocaru/elasticsearch-maven-plugin/pull/19
[3] https://github.com/yrodiere/hibernate-search/tree/HSEARCH-2434
Yoann Rodière <yoann(a)hibernate.org>
Hibernate NoORM Team
6:59 a.m.
Hi!
On Wed, Nov 2, 2016 at 12:38 PM, Yoann Rodiere <yoann(a)hibernate.org> wrote:
- And, perhaps most importantly, support for defining analyzers
in
elasticsearch.yml has been dropped. This means users have to resort to
the
index settings API to define their analyzers. So this breaks our
automatic
index creation / mapping generation feature: we put mappings just after
creating the index, but since the index was just created analyzer
definitions will be missing and the mapping will be rejected. See
HSEARCH-2434 (in the comments) for details.
The other important point (mentioned in the issue but I think it's
important to outline it here) concerning the analyzers is that they are now
defined per index: there are no global analyzers, you have to define them
for every index using the analyzer. It is totally different from our
current approach and make defining them manually (and maintaining them)
even more cumbersome. Thus I don't think we can consider defining them
manually as an option.
If we want to support Es 5, we need to provide a way to define these
analyzers in HS and make HS post them to Es.
--
Guillaume
8:59 a.m.
On 2 November 2016 at 12:59, Guillaume Smet <guillaume.smet(a)gmail.com>
wrote:
The other important point (mentioned in the issue but I think
it's
important to outline it here) concerning the analyzers is that they are now
defined per index: there are no global analyzers, you have to define them
for every index using the analyzer. It is totally different from our
current approach and make defining them manually (and maintaining them)
even more cumbersome. Thus I don't think we can consider defining them
manually as an option.
Yes, and that's the reason our automatic mapping generation is broken.
Since the scope of analyzers is now the index, posting analyzer definitions
should take place after index creation. But it should also happen before
mappings are put in the index (since they reference analyzers). In
practice, that means we must have users create indexes themselves (which
can't be done in bulk, which is cumbersome) and post analyzer definitions
themselves (which can be done in bulk and is relatively easy).
On 2 November 2016 at 12:59, Guillaume Smet <guillaume.smet(a)gmail.com>
wrote:
If we want to support Es 5, we need to provide a way to define these
analyzers in HS and make HS post them to Es.
Agreed: once we automate posting analyzer definitions to Elasticsearch from
HS, the actual API used to create indexes or post analyzers and its
"cumbersomeness" become irrelevant.
Our options are:
- Using the existing annotations and try as best we can to translate
Lucene analyzers to their name and parameters in ES: it won't help when
using non-Lucene analyzers, and it feels like something we can't maintain
forever.
- Introducing a new analyzer definition API: is it the right time, with
HS 6.0 looming over the horizon?
- Using the simple JSON file solution I proposed. Ok, maybe
*simplistic* rather
than *simple*, but it'd be easy to implement and probably wouldn't
require much maintenance work if we decide to keep it in 6.0.
Yoann Rodière <yoann(a)hibernate.org>
Hibernate NoORM Team
2928
days inactive
2928
days old
2 comments
2 participants
participants (2)
-
Guillaume Smet -
Yoann Rodiere