[hibernate-dev] Search: upgrade to Apache Lucene 5

[Sanne Grinovero]

Apache Lucene 5 is in candidate release now, and might be released
before the end of the month.
I've been testing it this weekend... and it's still Java7 compatible!

Sorry I got confused when I previously mentioned it would require
Java8: they did indeed switch "trunk" branch to require Java8, but
"trunk" is meant to become 6.0, while branch "5.0" was already
branched before that change in requirements.

So we could propose a timely update to Lucene 5 without necessarily
waiting for it to be a good time for Hibernate Search to upgrade to
Java8 / major release.

The initial "damage" is accounted as about 600 compile errors; I could
resolve approx 230 already as they were trivial.
The remaining ones require some more care & investigation, but seems
like we could get it done in 1-2 weeks, if we were to put our focus on
this...  it definitely looks better than the migration 3 -> 4.

Changes would again (obviously) affect users wherever they use
"native" Lucene APIs, but it seems like that this time we could do a
zero-changes in our APIs.. so we could do this in a minor release
without violating our "backwards compatibility policy" but I guess
that's arguable, as while our APIs would be "drop-in" compatible, the
user application wouldn't work with all the changes in Lucene code.
We would need to rewrite areas relating to:
 - Faceting
 - Filters and filter stacking
 - Custom Collectors (i.e. most code of Spatial)
 - FieldCache
 - Some IndexWriter code related to I/O errorhandling locking
 - Analyzer

A more extensive preview of changes is documented here:
http://people.apache.org/~anshum/staging_area/lucene-solr-5.0.0-RC1-rev1654615/lucene/changes/Changes.html#v5.0.0.documentation

We might want to start working soon on a new major already?
Whatever we do, we can't allow our users needing to wait as long as we
did for 4, especially since the upgrade is not as nasty.

Sanne