<div dir="ltr"><div class="gmail_extra">I don't see a lot of value in doing core-only releases. Releases are expensive because we have to update the website and documentation, and we have to announce the release everywhere. Releasing only the core won't change that.</div>
<div class="gmail_extra"><br></div><div class="gmail_extra"><div>Also, we don't try to maintain backwards compatibility between Alpha/Beta releases. So releasing only the core is only practical for minor/micro releases.<br>
</div><div><br></div></div><div class="gmail_extra">OTOH, doing a maven-only release is just a matter of starting the release script on the CI machine, and doing a couple clicks an hour later in the Nexus UI to release the staging repository. Doing a core-only maven-only release would have about the same overhead.<div>
<br></div><div>Wouldn't it be enough to move the Lucene directory to a separate repository (and release schedule)? We could easily do a couple maven-only releases to prepare for Search updgrades, I don't see any problems with that.</div>
<div><br></div><div>Cheers</div><div>Dan</div><div><br></div><div><br></div><div class="gmail_quote">On Wed, May 14, 2014 at 12:50 AM, Sanne Grinovero <span dir="ltr"><<a href="mailto:sanne@infinispan.org" target="_blank">sanne@infinispan.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">This is a reboot of the thread previously started on both the<br>
infinispan-dev and the hibernate-dev mailing list as "Handling of<br>
mutual dependency with Infinispan" [1].<br>
We discussed further during the Hibernate fortnightly meeting [2], and<br>
came to the conclusion that we need Infinispan to change how some<br>
repositories are organised and how the release is assembled.<br>
<br>
# The problem<br>
<br>
To restate the issue, as you might painfully remember, every time<br>
there is a need for a Lucene update or a Search update we need to sync<br>
up for a complex dance of releases in both projects to accommodate for<br>
a small-step iterative process to handle the circular dependency.<br>
This problem is not too bad today as since a year we're releasing the<br>
Lucene Directory in an unusual - and very unmaintainable - temporary<br>
solution to be compatible with two different major versions of Apache<br>
Lucene; namely what Infinispan Query needs and what Hibernate Search<br>
needs are different modules.<br>
But the party is over, and I want to finally drop support for Lucene 3<br>
and cleanup the unusual and unmaintainable build mess targeting a<br>
single Lucene version only.<br>
As soon as we converge to building a single version however - we're<br>
back to the complex problem we had when we supported a single version<br>
which is handling of a circular dependency - just that the problem has<br>
worsened lately the Lucene project has been more active and more<br>
inclined than what it used to be to break both internal and public<br>
APIs.<br>
<br>
In short, we have a circular dependency between Hibernate Search and<br>
Infinispan which we've been able to handle via hacks and some luck,<br>
but it imposes a serious threat to development flexibility, and the<br>
locked-in release process is not desirable either.<br>
<br>
# The solution<br>
<br>
we think in conclusion there's a single "proper" way out, and it also<br>
happens to provide some very interesting side effects in terms of<br>
maintenance overhead for everyone: Infinispan Core needs to release<br>
independently from the non-core modules.<br>
This would have the Lucene Directory depend on a released tag of<br>
infinispan-core, and be able to be released independently.<br>
Minor situations with benefit:<br>
- we often don't make any change in the Lucene Directory, still we<br>
need to release it.<br>
- when I actually need a release of it, I'm currently begging for a<br>
quick release of Infinispan: very costly<br>
The Big Ones:<br>
- we can manage the Lucene Directory to provide support for different<br>
versions of Lucene without necessarily breaking other modules<br>
- we can release quickly what's needed to move Search ahead in terms<br>
of Lucene versions without needing to make the Infinispan Query module<br>
compatible at the same time (in case you haven't followed this area:<br>
this seems to be my main activity rather than making valuable stuff).<br>
<br>
The goal is of course to linearise the dependencies; it seems to also<br>
simplify some of our tasks which is a welcome side-effect. I expect it<br>
also to make the project less scary for new contributors.<br>
<br>
# How does it impact users<br>
<br>
## Maven users<br>
modules will continue to be modules.. I guess nobody will notice,<br>
other than we might have a different versioning scheme, but we help<br>
people out via the Infinispan BOM.<br>
<br>
## Distribution users<br>
There should be no difference, other than (as well) some jars might<br>
not be aligned in terms of version. But that's probably even less of a<br>
problem, as I expect distribution users to just put what they get on<br>
their classpath.<br>
<br>
# How it impacts us<br>
<br>
1) I'll move the Lucene Directory project to an different repository;<br>
same for the Query related components.<br>
I think you should/could consider the same for other components, based<br>
on ad-hoc considerations of the trade offs, but I'd expect ultimately<br>
to see a more frequent and "core only" release.<br>
<br>
2) We'll have different kinds of releases: the "core only" and the<br>
"full releases".<br>
I think we'll also see components being released independently, but<br>
these are either Maven-only or meant for preparation of other<br>
components, or preparation for a "full release".<br>
<br>
3) Tests (!)<br>
Such a move should in no way relax the regression-safety of<br>
infinispan-core: we need to still consider it unacceptable for a core<br>
change to break one of the modules moving out of the main tree.<br>
Personally I think I've pushed many tests about problems found in the<br>
"query modules" as unit tests in core, so that should be relatively<br>
safe, but it also happened that someone would "tune" these.<br>
I realise it's not practical to expect people to run tests of<br>
downstream modules, so we'll have to automate most of these tasks in<br>
CI.<br>
Careful on perception: if today there are three levels of defence<br>
against a regression (the author, the reviewer and CI all running the<br>
suite for each change), in such an organisation you have only one. So<br>
ignoring a CI failure as a "probable hiccup" could be much more<br>
dangerous than usual.<br>
<br>
# When<br>
<br>
Doing this _might_ be a blocker for any Lucene update; so since one<br>
just happened I'll probably have no urgent need for a couple of weeks<br>
at least.<br>
But we shouldn't be in a position in which an update could not be<br>
possible, so I hope we'll agree to implement this sooner rather than<br>
later, so we won't have to do it during an emergency.<br>
<br>
Also while this might sound a bit crazy at first, I see many<br>
flexibility benefits which can't hurt now that the project is getting<br>
larger and more complex to release.<br>
Not least, having a micro release of "Infinispan essentials" would be<br>
very welcome in terms of lowing the initial barrier; this was proposed<br>
at various meetings and highly endorsed by many but just never<br>
happened.<br>
<br>
Any comment please? I hope I covered it all, and sorry for that :D<br>
<br>
Cheers,<br>
Sanne<br>
<br>
<br>
1 - <a href="http://lists.jboss.org/pipermail/hibernate-dev/2014-May/011419.html" target="_blank">http://lists.jboss.org/pipermail/hibernate-dev/2014-May/011419.html</a><br>
2 - <a href="http://transcripts.jboss.org/meeting/irc.freenode.org/hibernate-dev/2014/hibernate-dev.2014-05-13-13.24.log.html" target="_blank">http://transcripts.jboss.org/meeting/irc.freenode.org/hibernate-dev/2014/hibernate-dev.2014-05-13-13.24.log.html</a><br>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
</blockquote></div><br></div></div>