Required for HSEARCH-2434 if we don't want to drop support for ES 2.
Essentially, this ticket is about having an abstraction layer over the Elasticsearch services and metadata building in our hibernate-search-elasticsearch module. Implementations would be called "dialects", and would perform all work that has to be done differently from one version of ES to another. For instance:
* the "string" datatype disappeared in ES 5, where we have two distinct types: "text" and "keyword".
* the Delete-by-query operation is done through a plugin in ES 2, but through a core module in ES 5. And of course, the URLs are different.
* The "Optimize" API in ES 2 has been renamed "ForceMerge" in ES 2.1. We end up having "Optimize" available in ES 2.x (even 2.1+) but not 5.x, and "ForceMerge" available in ES 2.1+ (even 5.x) but not 2.0. And of course, the URLs are different.
* And much more. See the ticket description in HSEARCH-2434 for details.
h3. Problem 1: one can't have multiple versions of Jest in the classpath
Jest doesn't seem to support having multiple versions of it in the classpath (for instance Jest 1.x uses the same package names as Jest 2.x). Which means we cannot have a single -elasticsearch module that would automatically detect the dialect to use, because the version of Jest must be defined when compiling.
h3. Problem 2: Jest doesn't support ES 5 yet
Jest doesn't support ES 5 yet. It may work in most cases, but for instance the delete-by-query operation won't use the right URL (which changed in ES 5).
We could try to send a PR to add such support, but then we'd still have problem #1. Unless we manage to make the Jest project change package names from one version to another, which may not be easy.
We could also use the official Java REST client for Elasticsearch, which seems to support multiple Elasticsearch versions, but the API is very low-level: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/_performing_requests.html.
So we'd have to re-implement a lot of things that are already in Jest, most notably response parsing.On the other hand, it may be benificial: if we spot a bug or a missing feature, we'll only have to fix the issue ourselves with no reliance on another project's release train.
There are also another community client, [flummi|https://github.com/otto-de/flummi], but it also uses colliding package names for 2.x and 5.x.
There is a Java API for Elasticsearch, but we can't consider it because each version of this API only support connecting to ES nodes of the same version (e.g. the API in version 5.x doesn't support connecting to ES nodes in version 2.x): https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/client.html
Then there is the possibility to see a high-level, version-agnostic Elasticsearch client come to life: https://github.com/elastic/elasticsearch/tree/master/client/rest-high-level . But right now it hasn't been worked on much: https://github.com/elastic/elasticsearch/commits/master/client/rest-high-level
h3. Problem 3: our currently target a specific ES major
We sometimes send raw queries to ES, to initialize the metadata for instance. Since ES5 expects different metadata, we'll have to find a way to adapt our tests so that they use different raw queries depending on the version of ES...
h3. Problem 4: integration tests are currently configured to run against a single ES cluster
If we are to support multiple majors, we'll need to run the tests multiple times against multiple ES clusters (an ES 2.x cluster, then an ES 5.x cluster, etc.).
Maybe there is some way to do that with the maven-failsafe-plugin? Even then, we may have trouble making this work with the elasticsearch-maven-plugin. |
|
