Hi all, this one is a very desirable feature, yet tricky as there's high chances of ambiguity and confusion for end users. # Infinispan Remote indexing Infinispan embeds the Hibernate Search engine, and uses it to index data being inserted in any cache having indexing enabled. As you know Infinispan can be used to store Java POJOs, which get serialized using JBoss Marshalling - or encoded into Protobuf entries using Infinispan Protostream as helper layer. Hibernate OGM supports both modes, one meant for "Infinispan Embedded" and one for "Infinispan Remote" as that's what each encoding strategy is suited for. # Protobuf & indexing Protobuf is a well defined format with plenty of documentation which focuses on a "schema" of the encoding; Hibernate OGM is able to generate such schemas dynamically and will generate encoders and decoders which follow the encoding guidelines for Java objects. The meta schema of protobuf is not super flexible, yet there's the option of annotating the Protobuf schema elements using "annotations" in comments. Protostream allows inserting Hibernate Search annotations directly in these comments and will use them to generate the server side indexing configuration, implicitly also allowing such properties to be queried using indexed. For example you might have this string literally within the comments: "@Field(store = Store.YES, analyze = Analyze.YES)" A full example of schema can be found here [1]. (The Infinispan documentation is a bit sparse on this as they encourage people to use another code gen tool, best refer to tests as examples when working for OGM) # What should OGM users experience? A naive solution would be to allow people to use the Hibernate Search annotations on their JPA entities, and we have OGM copy these into the generated schema; there's a number of problems with that: - not all such annotations "translate" equally well [2] - there's a mismatch between JPA properties and underlying encoding fields - if I run a FullTextQuery do I expect it to work remotely? - what if I want to use Hibernate Search locally as well or instead? - references to local classes obviously won't work (custom fieldbridges, analyzers, etc..) An alternative is to look at these as "indexes" of the underlying store, so we'd use them to hint the Infinispan server about user provided hints such as those generated by `javax.persistence.Index`. I do think this is the cleaner approach, yet has two drawbacks: A- I guess ORM might implicitly generate some indexes in its metadata which the user might not have explicitly asked; e.g. accelerate unique constraints and foreign keys; it's possible these might not be as useful as expected in the Infinispan case. B- we won't be able to leverage the awesome full-text capabilities :-( I believe A is something we could ignore for now and revisit if there's actual demand. B is also not urgent, yet disappointing limitation as this capability is a distinguishing feature of this NoSQL. Would we agree that exposing such full-text capabilities would best be let to an ad-hoc backend in Hibernate Search 6? Thanks, Sanne 1 - http://blog.infinispan.org/2018/02/restful-queries-coming-to-infinispan-9... 2 - https://github.com/infinispan/infinispan/blob/master/remote-query/remote-... _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev

On 20 Mar 2018, at 12:41, Sanne Grinovero <sanne(a)hibernate.org&gt; wrote: On 20 March 2018 at 10:50, Emmanuel Bernard <emmanuel(a)hibernate.org <mailto:emmanuel@hibernate.org>> wrote: I think treating the client side HSearch input and transparently push it down is a recipe for disaster. Looks cool on paper until you have divergence between HSearch proper and Infinispan's or whatever Infinispan exposes. ​Looks like we agree on the premise, but you didn't comment on the proposal?​

So yes we could have GridDialect specific metadata information around indexing but I would not reuse Hibernate Search annotations for that. Treat Infinispan like any of the other GridDialect providers. As for the way to express FTQ, then Infinispan needs to make Ickle more powerful. Emmanuel On Wed 18-03-14 11:52, Sanne Grinovero wrote: Hi all, this one is a very desirable feature, yet tricky as there's high chances of ambiguity and confusion for end users. # Infinispan Remote indexing Infinispan embeds the Hibernate Search engine, and uses it to index data being inserted in any cache having indexing enabled. As you know Infinispan can be used to store Java POJOs, which get serialized using JBoss Marshalling - or encoded into Protobuf entries using Infinispan Protostream as helper layer. Hibernate OGM supports both modes, one meant for "Infinispan Embedded" and one for "Infinispan Remote" as that's what each encoding strategy is suited for. # Protobuf & indexing Protobuf is a well defined format with plenty of documentation which focuses on a "schema" of the encoding; Hibernate OGM is able to generate such schemas dynamically and will generate encoders and decoders which follow the encoding guidelines for Java objects. The meta schema of protobuf is not super flexible, yet there's the option of annotating the Protobuf schema elements using "annotations" in comments. Protostream allows inserting Hibernate Search annotations directly in these comments and will use them to generate the server side indexing configuration, implicitly also allowing such properties to be queried using indexed. For example you might have this string literally within the comments: "@Field(store = Store.YES, analyze = Analyze.YES)" A full example of schema can be found here [1]. (The Infinispan documentation is a bit sparse on this as they encourage people to use another code gen tool, best refer to tests as examples when working for OGM) # What should OGM users experience? A naive solution would be to allow people to use the Hibernate Search annotations on their JPA entities, and we have OGM copy these into the generated schema; there's a number of problems with that: - not all such annotations "translate" equally well [2] - there's a mismatch between JPA properties and underlying encoding fields - if I run a FullTextQuery do I expect it to work remotely? - what if I want to use Hibernate Search locally as well or instead? - references to local classes obviously won't work (custom fieldbridges, analyzers, etc..) An alternative is to look at these as "indexes" of the underlying store, so we'd use them to hint the Infinispan server about user provided hints such as those generated by `javax.persistence.Index`. I do think this is the cleaner approach, yet has two drawbacks: A- I guess ORM might implicitly generate some indexes in its metadata which the user might not have explicitly asked; e.g. accelerate unique constraints and foreign keys; it's possible these might not be as useful as expected in the Infinispan case. B- we won't be able to leverage the awesome full-text capabilities :-( I believe A is something we could ignore for now and revisit if there's actual demand. B is also not urgent, yet disappointing limitation as this capability is a distinguishing feature of this NoSQL. Would we agree that exposing such full-text capabilities would best be let to an ad-hoc backend in Hibernate Search 6? Thanks, Sanne 1 - http://blog.infinispan.org/2018/02/restful-queries-coming-to-infinispan-9... <http://blog.infinispan.org/2018/02/restful-queries-coming-to-infinispan-9... 2 - https://github.com/infinispan/infinispan/blob/master/remote-query/remote-... <https://github.com/infinispan/infinispan/blob/master/remote-query/remote-... _______________________________________________ hibernate-dev mailing list hibernate-dev(a)lists.jboss.org <mailto:hibernate-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/hibernate-dev <https://lists.jboss.org/mailman/listinfo/hibernate-dev>

On 22 Mar 2018, at 10:46, Sanne Grinovero <sanne(a)hibernate.org&gt; wrote: On 22 March 2018 at 07:41, Emmanuel Bernard <emmanuel(a)hibernate.org <mailto:emmanuel@hibernate.org>> wrote: > > On 20 Mar 2018, at 12:41, Sanne Grinovero <sanne(a)hibernate.org&gt; wrote: > > > > On 20 March 2018 at 10:50, Emmanuel Bernard <emmanuel(a)hibernate.org&gt; wrote: >> >> I think treating the client side HSearch input and transparently push it >> down is a recipe for disaster. Looks cool on paper until you have >> divergence between HSearch proper and Infinispan's or whatever >> Infinispan exposes. > > > Looks like we agree on the premise, but you didn't comment on the proposal? > > > > I did, it’s right below :) Granted that’s an option C. /The/ proposal in my first email is to have an Hibernate Search 6 "backend" to support Infinispan Remote. I see no comments on that, but it's ok since it looks like we agree on the problems I had already listed. Thanks, Sanne > > >> >> >> So yes we could have GridDialect specific metadata information around >> indexing but I would not reuse Hibernate Search annotations for that. >> Treat Infinispan like any of the other GridDialect providers. >> >> As for the way to express FTQ, then Infinispan needs to make Ickle more >> powerful. >> >> Emmanuel >> >> >> On Wed 18-03-14 11:52, Sanne Grinovero wrote: >>> >>> Hi all, >>> >>> this one is a very desirable feature, yet tricky as there's high >>> chances of ambiguity and confusion for end users. >>> >>> >>> # Infinispan Remote indexing >>> >>> Infinispan embeds the Hibernate Search engine, and uses it to index >>> data being inserted in any cache having indexing enabled. As you know >>> Infinispan can be used to store Java POJOs, which get serialized using >>> JBoss Marshalling - or encoded into Protobuf entries using Infinispan >>> Protostream as helper layer. >>> >>> Hibernate OGM supports both modes, one meant for "Infinispan Embedded" >>> and one for "Infinispan Remote" as that's what each encoding strategy >>> is suited for. >>> >>> >>> # Protobuf & indexing >>> >>> Protobuf is a well defined format with plenty of documentation which >>> focuses on a "schema" of the encoding; Hibernate OGM is able to >>> generate such schemas dynamically and will generate encoders and >>> decoders which follow the encoding guidelines for Java objects. >>> >>> The meta schema of protobuf is not super flexible, yet there's the >>> option of annotating the Protobuf schema elements using "annotations" >>> in comments. >>> Protostream allows inserting Hibernate Search annotations directly in >>> these comments and will use them to generate the server side indexing >>> configuration, implicitly also allowing such properties to be queried >>> using indexed. >>> >>> For example you might have this string literally within the comments: >>> "@Field(store = Store.YES, analyze = Analyze.YES)" >>> >>> A full example of schema can be found here [1]. >>> (The Infinispan documentation is a bit sparse on this as they >>> encourage people to use another code gen tool, best refer to tests as >>> examples when working for OGM) >>> >>> >>> # What should OGM users experience? >>> >>> A naive solution would be to allow people to use the Hibernate Search >>> annotations on their JPA entities, and we have OGM copy these into the >>> generated schema; there's a number of problems with that: >>> - not all such annotations "translate" equally well [2] >>> - there's a mismatch between JPA properties and underlying encoding >>> fields >>> - if I run a FullTextQuery do I expect it to work remotely? >>> - what if I want to use Hibernate Search locally as well or instead? >>> - references to local classes obviously won't work (custom >>> fieldbridges, analyzers, etc..) >>> >>> An alternative is to look at these as "indexes" of the underlying >>> store, so we'd use them to hint the Infinispan server about user >>> provided hints such as those generated by `javax.persistence.Index`. >>> I do think this is the cleaner approach, yet has two drawbacks: >>> A- I guess ORM might implicitly generate some indexes in its metadata >>> which the user might not have explicitly asked; e.g. accelerate unique >>> constraints and foreign keys; it's possible these might not be as >>> useful as expected in the Infinispan case. >>> B- we won't be able to leverage the awesome full-text capabilities :-( >>> >>> I believe A is something we could ignore for now and revisit if >>> there's actual demand. >>> >>> B is also not urgent, yet disappointing limitation as this capability >>> is a distinguishing feature of this NoSQL. Would we agree that >>> exposing such full-text capabilities would best be let to an ad-hoc >>> backend in Hibernate Search 6? >>> >>> >>> Thanks, >>> Sanne >>> >>> 1 - >>> http://blog.infinispan.org/2018/02/restful-queries-coming-to-infinispan-9... >>> 2 - >>> https://github.com/infinispan/infinispan/blob/master/remote-query/remote-... >>> _______________________________________________ >>> hibernate-dev mailing list >>> hibernate-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev