[hibernate-dev] [Hibernate Search] Donating some of our source code to Infinispan

[Emmanuel Bernard]

I mean discoverable by humans. i.e. easly seeing that the feature even exists. I could not care less about pluggable lookup strategies ;)

> On 26 Jan 2015, at 17:08, Sanne Grinovero <sanne at hibernate.org> wrote:
> 
> On 26 January 2015 at 15:05, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
>> One angle that is a bit unclear to me is how easy is this consumable and discoverable by our users.
> 
> Discovery would be the same as today, same configuration property.
> We have your "resolution strategy" which translates from short name to
> fully qualified name, which is already "pluggable" in the case of the
> IndexManager lookup. So we have plenty of options A) look for the new
> FQCN B) plug a custom resolver strategy for the DirectoryProvider
> names C)use the custom IndexManager: we had already decided at our f2f
> meeting that it would be far easier for people to only have to choose
> the IndexManager implementation, which would be in charge of setting
> up the right backend/directory couples as a "profile".
> 
> WildFly users don't even need to know about changes in the module
> names / jar names / maven coordinates as the instructions would still
> be: download both modules, drop them in your AS -> works.
> 
> In short, we can keep user changes to minimal.
> 
>> Some in this thread mentioned creating a separate repository. I see this as a massive ease of use problem if the user is exposed to finding the right version of each of these three libs. Likewise for documentation, project identity, etc.
> 
> +1 not my preferred choice either. I think we'll stick to my plan, at
> least until I'm proven this proposal is utterly broken.
> 
>> I think the merge of this piece of code into Infinispan does not create too much of a problem in that regard. The main issue is that as an Hibernate Search user, I don’t know which Infinispan to target. And an Infinispan user does not really care for the most part about an explicit HSearch usage outside Infinispan query.
> 
> We'll have to document that. Indeed quite tricky, as there might be
> cases in which there is no compatible version of Infinispan whatsoever
> (for example each time we migrate to a new Lucene there would be a
> gap).. but my hope is that would be some Alpha/Beta, and by the time
> of a Final release we'd have a viable suggestion, with more options
> while the branch matures in maintenance mode.
> 
>> BTW will the documentation on HSearch integration with Infinispan Lucene Directory remain in HSearch’s doc or be copied in Infinispan’s?
> 
> I'd prefer to have a single consistent document for our users to read:
> we'd have the high level description from the point of view of an
> Hibernate user, and exposing to the subset of tuning options which
> make sense to him.
> Infinispan should have a different copy, especially as it's becoming
> clear that tuning and configuration for this specific component is
> quite different in the two different use cases. Ultimately both
> "engines" should be able to guess and auto-configure several options
> which are manual today, and these guessing and custom backends are
> different.
> The low level details of the Directory tuning should probably live in
> Infinispan only, in close coupling to the code changes, to make sure
> there is a document set kept in synch with the specific version. From
> experience we know the new configuration options needed by Infinispan
> are rarely in the "need to know" for Hibernate users.
> 
> Sanne
> 
>> 
>> Emmanuel
>> 
>>> On 25 Jan 2015, at 22:19, Sanne Grinovero <sanne at hibernate.org> wrote:
>>> 
>>> All,
>>> we've discussed several times the issues we have because of the
>>> circular dependency between Hibernate Search and Infinispan.
>>> Normally the pain point we aim to address with such discussions is the
>>> need to carefully coordinate between releases [of Search and
>>> Infinispan] in our quest for a final stable release, and brings some
>>> release pressure.
>>> 
>>> More recently it has also become a compatibility problem, as the two
>>> projects target different platforms/environments, and have different
>>> life-cycle expectations - creating more maintenance work such as lots
>>> of back-porting of patches.. distracting from our goals.
>>> 
>>> We already discussed some solutions, but none too convincing. I have a
>>> new fresh proposal which I feel is more interesting:
>>> 
>>> # New plan
>>> 
>>> 1) the module "/infinispan" from the Hibernate Search source tree is
>>> moved to the Infinispan project into some "hibernate-search-directory"
>>> Maven module.
>>> 
>>> 2) Hibernate Search drops any dependency to Infinispan
>>> 
>>> Reminder: this "/infinispan" module we have contains just a couple of
>>> classes, and represents the "DirectoryProvider" implementation which
>>> integrates with the Hibernate Search autodiscovery and creates a
>>> Directory instance (whose implementation always lived in Infinispan).
>>> Most of the code is about applying configuration properties,
>>> integration tests.
>>> 
>>> It's a very simple plan, but has some valuable consequences:
>>> - Search can move on without ever needing to wait for Infinispan -
>>> and vice versa.
>>> - There is no longer a circular dependency
>>> - Search would be in control of its Lucene dependency, and can
>>> upgrade as needed. We could experiment with different Lucene versions
>>> without necessarily wait for Infinispan to solve compatiblity issues
>>> first - and often more complex as that means then for Infinispan to be
>>> able to guarantee a compatibility with a *range* of Lucene versions,
>>> to include both the target Lucene and the version currently consumed
>>> via Infinispan Query / Hibernate Search Engine.
>>> - Infinispan can *opt* to stick with an older version of Search - or
>>> update - provided it can satisfy both a) integration with possible
>>> changes to the Search SPI b) an update for possible new requirements
>>> of the new Lucene version (the Lucene Directory might need
>>> compatibility fixes)
>>> - The dependency structure would better match the one as provided by
>>> our Enterprise Products. For example, it's the JDG distribution - not
>>> Hibernate Search - to provide these integration bits, so it makes more
>>> sense to build the Directory against the specific version of
>>> Infinispan than against the specific version of Search.
>>> - It will be easier to have new Infinispan code take advantage of
>>> features exposed on the Infinispan Lucene Directory - if all changed
>>> parts are in the same [Infinispan] repository. This is actually being
>>> a problem right now, as it's holding back a POC meant to deliver great
>>> improvements with the Directory implementation.
>>> 
>>> And not least we'll have faster builds ;-)
>>> 
>>> # Drawbacks
>>> 
>>> First one is we'll probably have our users need to change some details
>>> of how they build.
>>> Infinispan might need to fix some SPI related code before being able
>>> to upgrade, but historically our SPI has been extremely stable.
>>> The real problem is that when such a thing happens, after we've
>>> released a version of Search there might be some time before a
>>> compatible version of Infinispan is made available as well.
>>> 
>>> In practice this means the gap of time in which we have to catch up on
>>> API changes is "exposed" to end users wanting to use our latest and
>>> possibly blocked - but while they would then see the tip of the
>>> iceberg of our integration, I believe it would still take the same
>>> amount of waiting time in terms of calendar dates in which the working
>>> duo is available to them - as with the current model in such a
>>> situation we need to wait for the same Infinispan release to happen
>>> before we can release ours.
>>> So: same time, but we'd have a leaner process, and possibly quicker
>>> releases for all users not interested in that - or just benefits in
>>> all those scenarios in which we don't break APIs which is very common.
>>> I've not identified other problems, so my opinion is that these are
>>> well worth the benefits.
>>> 
>>> # Consequences for our users
>>> 
>>> Not much. Even today we expect our users to depend on several jar
>>> files provided by the Infinispan team; this would be just one more.
>>> Opens some questions though:
>>> 
>>> A) Should the Maven group id be changed? I'd expect it to be
>>> transferred to "org.infinispan" group at least, and probably need a
>>> better artifact id too.
>>> 
>>> B) License. Our code is LGPL, most of Infinispan is ASL - but not all
>>> of it. So I expect it would be possible to keep the existing license
>>> at least for now, and defer eventual license changes as a separate
>>> step (if people feel need for any change at all).
>>> 
>>> C) Documentation. Besides the needed updates in Maven coordinates /
>>> download sources, I don't expect much of a difference: we'd still
>>> explain how to set this integration up.
>>> 
>>> D) Distribution. Today we distribute this module, and its
>>> dependencies, in our release bundles. Which implies we distribute a
>>> copy of various Infinispan jars.
>>> I think we should drop these from our distribution - even though it
>>> might seem counter-intuitive:
>>> while it might seem convenient to have these included, the whole point
>>> of the change would be that there would be more flexibility in which
>>> versions of Infinispan would work with Search. And actually the
>>> integration tests and this specific knowledge would be responsibility
>>> of Infinispan.
>>> 
>>> Am I failing to see a more critical issue?
>>> How would you all feel about our code being transferred to the
>>> different project?
>>> 
>>> Sanne
>>> _______________________________________________
>>> hibernate-dev mailing list
>>> hibernate-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>