While reviewing ISPN-200 it's clear that Israel needed to copy over some code from Hibernate Search in Infinispan Query, to make some small changes to internal classes; All the changes I spotted so far are around making some objects serializable. I don't want to keep these copies around, especially as we're talking about very core and complex classes as org.hibernate.search.query.engine.impl.HSQueryImpl, which is not going to be maintainable in a forked situation, and definitely not meant for consumption by other projects. So I have some options: 1) I change the HSQueryImpl in Search to make it serializable, porting Israel's patches "upstream". 2) Create an externalizer in Infinispan Query which knows about the internals of HSQueryImpl Clearly the Externalizer solution is highly coupled to the specific version of Search, so that might become another issue in maintaining them aligned, and this is not the appropriate class to be bound to a "serialization contract", but is also the recommended way according to Infinispan's spirit (compared to make it Serializable). We could add a submodule in Hibernate Search to specifically test that the Query module is not going to break, but this is going to create a dangerous embrace in the release cycles of the two projects. I'm starting to wonder if Query shouldn't live in the Hibernate Search repository: it's highly coupled to Search, but nicely decoupled from Infinispan as it depends on it's public API (so still subject to be broken by changes in Infinispan, but that would be considered a bug in Infinispan). It is of course possible to polish the Search APIs more to make sure that it's also nicely decoupled from it, but it's harder to achieve; specifically by the very recurrent need to serialize/marshal it's objects, or specific Lucene classes. Hibernate Search has always been compatible with a specific version of Lucene only, as this API is in continous flux too, so one of the goals of the project is to "shield" users from the breaking changes which are often introduced; to be conidered as well: most classes in Lucene are not serializable, or even if they are it's a deprecated feature and the maintainers are not interested in keeping wire compatibility. of course in the short term it's easy to make those classes serializable, or not even needed just defining an externalizer. So I think we should start with one of these while this complex contribution takes shape, but also consider options for near-future. opinions? Cheers, Sanne _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Back from the dead

On Jun 13, 2011, at 8:25 PM, Sanne Grinovero wrote: > While reviewing ISPN-200 it's clear that Israel needed to copy over > some code from Hibernate Search in Infinispan Query, to make some > small changes to internal classes; All the changes I spotted so far > are around making some objects serializable. > I don't want to keep these copies around, especially as we're talking > about very core and complex classes as > org.hibernate.search.query.engine.impl.HSQueryImpl, which is not going > to be maintainable in a forked situation, and definitely not meant for > consumption by other projects. > > So I have some options: > 1) I change the HSQueryImpl in Search to make it serializable, porting > Israel's patches "upstream". > 2) Create an externalizer in Infinispan Query which knows about the > internals of HSQueryImpl I don't see a big problem with 1. I guess it's even recommended for people using faceting and keeping the query around.

> Clearly the Externalizer solution is highly coupled to the specific > version of Search, so that might become another issue in maintaining > them aligned, and this is not the appropriate class to be bound to a > "serialization contract", but is also the recommended way according to > Infinispan's spirit (compared to make it Serializable). why is it the recommended way. We're not putting HSQueryImpl in the grid are we? What's the need behind serialization?

> We could add a submodule in Hibernate Search to specifically test that > the Query module is not going to break, but this is going to create a > dangerous embrace in the release cycles of the two projects. > > I'm starting to wonder if Query shouldn't live in the Hibernate Search > repository: it's highly coupled to Search, but nicely decoupled from > Infinispan as it depends on it's public API (so still subject to be > broken by changes in Infinispan, but that would be considered a bug in > Infinispan). I'm fine with that if you think it's best but I suspect that will be a bit awkward for users.

> It is of course possible to polish the Search APIs more to make sure > that it's also nicely decoupled from it, but it's harder to achieve; > specifically by the very recurrent need to serialize/marshal it's > objects, or specific Lucene classes. Hibernate Search has always been > compatible with a specific version of Lucene only, as this API is in > continous flux too, so one of the goals of the project is to "shield" > users from the breaking changes which are often introduced; to be > conidered as well: most classes in Lucene are not serializable, or > even if they are it's a deprecated feature and the maintainers are not > interested in keeping wire compatibility. > > of course in the short term it's easy to make those classes > serializable, or not even needed just defining an externalizer. So I > think we should start with one of these while this complex > contribution takes shape, but also consider options for near-future. +1

On Jun 30, 2011, at 11:50 AM, Sanne Grinovero wrote: >>> Clearly the Externalizer solution is highly coupled to the specific >>> version of Search, so that might become another issue in maintaining >>> them aligned, and this is not the appropriate class to be bound to a >>> "serialization contract", but is also the recommended way according to >>> Infinispan's spirit (compared to make it Serializable). >> >> why is it the recommended way. We're not putting HSQueryImpl in the grid are we? What's the need behind serialization? > > Yes we are sending it across the cluster, as it contains all the state > for the Query (both original definition and in-flight progress) and > then carries over the partial state to the next node to properly > iterate on the results. What concerns me most is the need to be > tightly coupled to this very internal class, as it also happens to be > the core of Search and will likely be affected by changes across > versions. > We'll see in a second phase if we can instead extract the needed state > and re-create it remotely from it: we might need to add more accessors > to the HSQuery interface but at least we'll have an interface coupling > instead of a binary-format coupling. Worse case, we do expose a HSQuery contract for hydratation / dehydratation so that an externalizer has a clear contract.