Currently, we are testing the Elasticsearch backend by executing tests extracted from test-jars created in both -engine and -orm.

That works great most of the time, but we do have some issues related to transitive dependencies not being handled by Maven for test-jars (see https://maven.apache.org/plugins/maven-jar-plugin/examples/create-test-jar.html).

The most straightforward solution would be to move all the tests we want to execute on multiple backends to separate projects that would be specifically designed to allow their execution against a backend specified as a parameter (think of this as some kind of TCK).
But since it's a rather drastic move, we'd have to discuss this first.
Note that in some cases, if we don't want to move the tests, we could also try reversing the dependencies, configuring tests in -orm for instance to execute once for each backend.