[hibernate-dev] [Hibernate Search] Donating some of our source code to Infinispan

[Sanne Grinovero]

All,
we've discussed several times the issues we have because of the
circular dependency between Hibernate Search and Infinispan.
Normally the pain point we aim to address with such discussions is the
need to carefully coordinate between releases [of Search and
Infinispan] in our quest for a final stable release, and brings some
release pressure.

More recently it has also become a compatibility problem, as the two
projects target different platforms/environments, and have different
life-cycle expectations - creating more maintenance work such as lots
of back-porting of patches.. distracting from our goals.

We already discussed some solutions, but none too convincing. I have a
new fresh proposal which I feel is more interesting:

# New plan

1) the module "/infinispan" from the Hibernate Search source tree is
moved to the Infinispan project into some "hibernate-search-directory"
Maven module.

2) Hibernate Search drops any dependency to Infinispan

Reminder: this "/infinispan" module we have contains just a couple of
classes, and represents the "DirectoryProvider" implementation which
integrates with the Hibernate Search autodiscovery and creates a
Directory instance (whose implementation always lived in Infinispan).
Most of the code is about applying configuration properties,
integration tests.

It's a very simple plan, but has some valuable consequences:
 - Search can move on without ever needing to wait for Infinispan -
and vice versa.
 - There is no longer a circular dependency
 - Search would be in control of its Lucene dependency, and can
upgrade as needed. We could experiment with different Lucene versions
without necessarily wait for Infinispan to solve compatiblity issues
first - and often more complex as that means then for Infinispan to be
able to guarantee a compatibility with a *range* of Lucene versions,
to include both the target Lucene and the version currently consumed
via Infinispan Query / Hibernate Search Engine.
 - Infinispan can *opt* to stick with an older version of Search - or
update - provided it can satisfy both a) integration with possible
changes to the Search SPI b) an update for possible new requirements
of the new Lucene version (the Lucene Directory might need
compatibility fixes)
 - The dependency structure would better match the one as provided by
our Enterprise Products. For example, it's the JDG distribution - not
Hibernate Search - to provide these integration bits, so it makes more
sense to build the Directory against the specific version of
Infinispan than against the specific version of Search.
 - It will be easier to have new Infinispan code take advantage of
features exposed on the Infinispan Lucene Directory - if all changed
parts are in the same [Infinispan] repository. This is actually being
a problem right now, as it's holding back a POC meant to deliver great
improvements with the Directory implementation.

And not least we'll have faster builds ;-)

# Drawbacks

First one is we'll probably have our users need to change some details
of how they build.
Infinispan might need to fix some SPI related code before being able
to upgrade, but historically our SPI has been extremely stable.
The real problem is that when such a thing happens, after we've
released a version of Search there might be some time before a
compatible version of Infinispan is made available as well.

In practice this means the gap of time in which we have to catch up on
API changes is "exposed" to end users wanting to use our latest and
possibly blocked - but while they would then see the tip of the
iceberg of our integration, I believe it would still take the same
amount of waiting time in terms of calendar dates in which the working
duo is available to them - as with the current model in such a
situation we need to wait for the same Infinispan release to happen
before we can release ours.
So: same time, but we'd have a leaner process, and possibly quicker
releases for all users not interested in that - or just benefits in
all those scenarios in which we don't break APIs which is very common.
I've not identified other problems, so my opinion is that these are
well worth the benefits.

# Consequences for our users

Not much. Even today we expect our users to depend on several jar
files provided by the Infinispan team; this would be just one more.
Opens some questions though:

 A) Should the Maven group id be changed? I'd expect it to be
transferred to "org.infinispan" group at least, and probably need a
better artifact id too.

 B) License. Our code is LGPL, most of Infinispan is ASL - but not all
of it. So I expect it would be possible to keep the existing license
at least for now, and defer eventual license changes as a separate
step (if people feel need for any change at all).

 C) Documentation. Besides the needed updates in Maven coordinates /
download sources, I don't expect much of a difference: we'd still
explain how to set this integration up.

 D) Distribution. Today we distribute this module, and its
dependencies, in our release bundles. Which implies we distribute a
copy of various Infinispan jars.
I think we should drop these from our distribution - even though it
might seem counter-intuitive:
while it might seem convenient to have these included, the whole point
of the change would be that there would be more flexibility in which
versions of Infinispan would work with Search. And actually the
integration tests and this specific knowledge would be responsibility
of Infinispan.

Am I failing to see a more critical issue?
How would you all feel about our code being transferred to the
different project?

Sanne