Regarding problem 1 & 2: Jest

• since Jest doesn't support ES5, I guess we could have - for now - Jest to connect to an ES2 cluster, and a different one for ES5+
• we agreed that auto-detection of the ES version is desirable, but we can make this a later feature. For example we could say it's only going to work for ES5+ version, on the new client.

Problem 3:

• we'll have to duplicate some tests, and enable/disable the right one with annotations. Not too different than what we do with Hibernate ORM dialects: some of them require a specific DB.
• hopefully we won't need to duplicate too many tests?

Problem 4:

• we could pick a "favourite" version which is going to run all tests locally by default, and then have CI report on combinations. Similar to what we do with ORM dialects again.

I think the main think to decide is how to implement those "ES dialects", like the JIRA title suggests. My vote goes to use the driver which the ES team is maintaining, I'm confident they would be responsive on our feedback as we make progress.
As you mentioned, there is no much activity currently on the "high level" version. I'd use the low level one? Might be more work, but I expec also it could give us more flexibility and push performance higher, for example reusing encoded blocks and trying to avoid some runtime rendering of JSON structures.