Some other projects use Hibernate Search - specifically the engine module - to bridge their domain model to a Lucene index and take advantage of its high performance low-level integration with Lucene. This is generally achieved by indexing a "valueholder" object which has most logic to create a custom o.a.l.Document in a top level @ClassBridge, or by using some custom FieldBridges, but has some strong limitations and feels more like a hack.

In Hibernate Search 5.0 we will make this more flexible, so to move away from an annotated-entities index engine only to make it easier to index objects whose "schema" is more flexible than the contraints imposed by the staticness of a compiled Java class.

In particular what seems to be troublesome is to implement multiple "user types" using the same type (java type) as value holders:

So this first refactoring [HSEARCH-1402] will be about moving the *internal* contract - without changing public APIs - to use a different identification than the current Class when we lookup for indexing information for a specific indexed type.

Hi, I'm beginning to wonder which of these things we actually do need in 4.4 to satisfy the requirements of ISPN remote queries. At the moment we have: * the ability to index "user types" via one generic "value holder type" using a smart class bridge which adds all the required fields; with HSEARCH-1396 [1] we allow for passing bridge _instances_ to the engine which e.g. enables to inject the user type configuration (which fields required) into this class bridge by the client * the ability to query the properties of a user type by allowing to specify the bridges to be used for given fields (HSEARCH-1409 [2], PR is opened [3]); In particular this allows to query against fields which are not statically declared for the value holder type > the trouble is to filter on the correct user type where the Hibernate Search APIs expect - as basic filtering strategy - a java Class instance, or simply to have the flexibility to configure separate backends for each user type. I think this part is still missing. I'm wondering though whether it couldn't be emulated "well enough" by making the ProjectionConstants.OBJECT_CLASS field user-writable (as suggested with HSEARCH-1402) and querying on this field? Together with the dynamic sharding work from Hardy also the "separate backends for each user type" part might be addressed by assigning each user type to a separate shard of the value holder type's index. If this is feasible, it would allow to stick to the class-based design for the time being. This might help with limiting the number of changes required to be delivered with 4.4 and focus on implementing the dynamic entity feature properly in HSearch 5. Any thoughts? --Gunnar [1] https://hibernate.atlassian.net/browse/HSEARCH-1396 [2] https://hibernate.atlassian.net/browse/HSEARCH-1409 [3] https://github.com/hibernate/hibernate-search/pull/486 2013/9/11 Sanne Grinovero <sanne(a)hibernate.org&gt; > Some other projects use Hibernate Search - specifically the engine > module - to bridge their domain model to a Lucene index and take > advantage of its high performance low-level integration with Lucene. > This is generally achieved by indexing a "valueholder" object which > has most logic to create a custom o.a.l.Document in a top level > @ClassBridge, or by using some custom FieldBridges, but has some > strong limitations and feels more like a hack. > > In Hibernate Search 5.0 we will make this more flexible, so to move > away from an annotated-entities index engine only to make it easier to > index objects whose "schema" is more flexible than the contraints > imposed by the staticness of a compiled Java class. > > We've discussed however to introduce some of these features right now > (in version 4.4), so that we can start exploring the direction > gradually and get some early feedback. In particular what seems to be > troublesome is to implement multiple "user types" using the same type > (java type) as valueholders: the trouble is to filter on the correct > user type where the Hibernate Search APIs expect - as basic filtering > strategy - a java Class instance, or simply to have the flexibility to > configure separate backends for each user type. > > So this first refactoring [HSEARCH-1402] will be about moving the > *internal* contract - without changing public APIs - to use a > different identification than the current Class when we lookup for > indexing information for a specific indexed type. > > Comments very welcome. > > Sanne > > - https://hibernate.atlassian.net/browse/HSEARCH-1379 Explicit support > for indexing free-form (dynamic) entities > > - https://hibernate.atlassian.net/browse/HSEARCH-1402 Allow explicit > user control of ProjectionConstants.OBJECT_CLASS value during indexing > > - https://hibernate.atlassian.net/browse/HSEARCH-1404 More flexibility > than just Class to identify indexing metadata > _______________________________________________ > hibernate-dev mailing list > hibernate-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev >

Hi Gunnar, yes we've been reviewing the Infinispan Query code and it seem that what we have today if good enough for this release cycle. We're keeping the issues open as the current approach can use some polish, but time-wise we need to make space for more experimentation.

Sanne On 7 October 2013 09:20, Gunnar Morling <gunnar(a)hibernate.org&gt; wrote: > Sanne, > > Is there any news on where we stand with this? > > I saw you discussed and moved some issues to HSEARCH 5; Does that mean we > have everything in place now for the ISPN use case? I'd have assumed at > least making the OBJECT_CLASS (HSEARCH-1402) field user-writable would be > needed for restricting queries to specific "user types", but this one has > been moved to 5 as well. > > Thanks, > > --Gunnar > > > > 2013/9/20 Gunnar Morling <gunnar(a)hibernate.org&gt; >> >> Hi, >> >> I'm beginning to wonder which of these things we actually do need in 4.4 >> to satisfy the requirements of ISPN remote queries. At the moment we have: >> >> * the ability to index "user types" via one generic "value holder type" >> using a smart class bridge which adds all the required fields; with >> HSEARCH-1396 [1] we allow for passing bridge _instances_ to the engine which >> e.g. enables to inject the user type configuration (which fields required) >> into this class bridge by the client >> >> * the ability to query the properties of a user type by allowing to >> specify the bridges to be used for given fields (HSEARCH-1409 [2], PR is >> opened [3]); In particular this allows to query against fields which are not >> statically declared for the value holder type >> >> > the trouble is to filter on the correct user type where the Hibernate >> > Search APIs expect - as basic filtering strategy - a java Class instance, or >> > simply to have the flexibility to configure separate backends for each user >> > type. >> >> I think this part is still missing. I'm wondering though whether it >> couldn't be emulated "well enough" by making the >> ProjectionConstants.OBJECT_CLASS field user-writable (as suggested with >> HSEARCH-1402) and querying on this field? Together with the dynamic sharding >> work from Hardy also the "separate backends for each user type" part might >> be addressed by assigning each user type to a separate shard of the value >> holder type's index. >> >> If this is feasible, it would allow to stick to the class-based design for >> the time being. This might help with limiting the number of changes required >> to be delivered with 4.4 and focus on implementing the dynamic entity >> feature properly in HSearch 5. Any thoughts? >> >> --Gunnar >> >> [1] https://hibernate.atlassian.net/browse/HSEARCH-1396 >> [2] https://hibernate.atlassian.net/browse/HSEARCH-1409 >> [3] https://github.com/hibernate/hibernate-search/pull/486 >> >> >> 2013/9/11 Sanne Grinovero <sanne(a)hibernate.org&gt; >>> >>> Some other projects use Hibernate Search - specifically the engine >>> module - to bridge their domain model to a Lucene index and take >>> advantage of its high performance low-level integration with Lucene. >>> This is generally achieved by indexing a "valueholder" object which >>> has most logic to create a custom o.a.l.Document in a top level >>> @ClassBridge, or by using some custom FieldBridges, but has some >>> strong limitations and feels more like a hack. >>> >>> In Hibernate Search 5.0 we will make this more flexible, so to move >>> away from an annotated-entities index engine only to make it easier to >>> index objects whose "schema" is more flexible than the contraints >>> imposed by the staticness of a compiled Java class. >>> >>> We've discussed however to introduce some of these features right now >>> (in version 4.4), so that we can start exploring the direction >>> gradually and get some early feedback. In particular what seems to be >>> troublesome is to implement multiple "user types" using the same type >>> (java type) as valueholders: the trouble is to filter on the correct >>> user type where the Hibernate Search APIs expect - as basic filtering >>> strategy - a java Class instance, or simply to have the flexibility to >>> configure separate backends for each user type. >>> >>> So this first refactoring [HSEARCH-1402] will be about moving the >>> *internal* contract - without changing public APIs - to use a >>> different identification than the current Class when we lookup for >>> indexing information for a specific indexed type. >>> >>> Comments very welcome. >>> >>> Sanne >>> >>> - https://hibernate.atlassian.net/browse/HSEARCH-1379 Explicit support >>> for indexing free-form (dynamic) entities >>> >>> - https://hibernate.atlassian.net/browse/HSEARCH-1402 Allow explicit >>> user control of ProjectionConstants.OBJECT_CLASS value during indexing >>> >>> - https://hibernate.atlassian.net/browse/HSEARCH-1404 More flexibility >>> than just Class to identify indexing metadata >>> _______________________________________________ >>> hibernate-dev mailing list >>> hibernate-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> >> >

On 7 October 2013 10:51, Gunnar Morling <gunnar(a)hibernate.org&gt; wrote: > 2013/10/7 Sanne Grinovero <sanne(a)hibernate.org&gt; >> >> Hi Gunnar, >> yes we've been reviewing the Infinispan Query code and it seem that >> what we have today if good enough for this release cycle. >> We're keeping the issues open as the current approach can use some >> polish, but time-wise we need to make space for more experimentation. > > > Ok, that sounds good. How will queries restricted to one user type be > realized with the current implementation? The current implementation doesn't expose any Lucene API, it's all nicely hidden by a super-easy high level filtering API. The drawback is that far less features are available to the end user, but the advantage is that the Lucene Query which is internally generated is never exposed, so fully controlled by the Infinispan parser. This is the relevant line: https://github.com/infinispan/infinispan/blob/6.0.0.CR1/remote-query/remo... Sanne