[Red Hat JIRA] (ISPN-12419) Expose indexed-embedding as a separate annotation in Protobuf index mapping
by Yoann Rodière (Jira)
[ https://issues.redhat.com/browse/ISPN-12419?page=com.atlassian.jira.plugi... ]
Yoann Rodière updated ISPN-12419:
---------------------------------
Description:
This ticket is about a behavior change between Infinispan 11 and 12 caused by the upgrade to Hibernate Search 6; it must be addressed before the release of Infinispan 12.
In Infinispan 11, when mapping a Protobuf type to an index, one must use the {{@Field}} annotation in order to mark a composite type as "embedded", e.g.:
{code}
message User {
/**
* @Field(store = Store.YES)
* @SortableField
*/
required int32 id = 1;
/**
* @Field(store = Store.YES)
*/
optional User manager = 2;
}
{code}
The {{@Field}} annotation on {{manager}} does not actually create a field: instead, it specifies that all fields defined in {{manager}} should be embedded in {{User}}. In this case, it means user will have a field named {{manager.id}}.
One obvious problem is that we're hijacking an annotation to express something really different from what it was intended for. It would arguably be better to have a dedicated annotation for that kind of embedding, similar to the {{@IndexedEmbedded}} annotation from Hibernate Search.
But a less obvious and much more problematic consequence of that choice is that we are forced to put implicit limits on recursive models, and users cannot control these limits. Indeed, Infinispan 12 only uses static metamodels when defining Hibernate Search indexes, so infinite recursion is simply not an option. In the example above, we have to declare all fields at bootstrap, and that includes the recursions such as {{manager.id}}, {{manager.manager.id}}, {{manager.manager.manager.id}}, etc. Currently we limit embedding depth to {{7}}, which is very arbitrary.
To solve _that_ problem, we have two alternatives:
# Embrace infinite recursion, meaning we will define a dynamic schema. In the example above, we would define static fields {{manager}}, {{manager.id}} and {{manager.manager}}, and inside {{manager}} we would define dynamic fields for path patterns {{\*.manager}} (object field) and {{\*.manager.id}} (Integer field, stored, sortable). The main problem with this behavior is there is a possibility that user schemas introduce conflicts (e.g. two definitions of "manager" in two different embedded types), which will probably lead to runtime errors or non-matching queries.
# Switch to a dedicated annotation to configure embedding, e.g. {{@IndexedEmbedded}}. This annotation could be unlimited by default. Hibernate Search will throw an exception if that leads infinite recursion, and when that happens, users will be able to set attributes on the annotation to control recursion, for example with {{@IndexedEmbedded(includePaths = <finite list of field paths to embed>)}} or {{@IndexedEmbedded(includeDepth = <depth of recursion when indexing>)}}.
I'd be in favor of solution 2, which is the solution we implement in the Hibernate Search / Hibernate ORM mapper : https://docs.jboss.org/hibernate/search/6.0/reference/en-US/html_single/#...
Judging from a previous conversation, it seems [~anistor] [agrees|https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/...].
See also the full conversation, starting here: https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/topic/I...
was:
This ticket is about a behavior change between Infinispan 11 and 12 caused by the upgrade to Hibernate Search 6; it must be addressed before the release of Infinispan 12.
In Infinispan 11, when mapping a Protobuf type to an index, one must use the {{@Field}} annotation in order to mark a composite type as "embedded", e.g.:
{code}
message User {
/**
* @Field(store = Store.YES)
* @SortableField
*/
required int32 id = 1;
/**
* @Field(store = Store.YES)
*/
optional User manager = 2;
}
{code}
The {{@Field}} annotation on {{manager}} does not actually create a field: instead, it specifies that all fields defined in {{manager}} should be embedded in {{User}}. In this case, it means user will have a field named {{manager.id}}.
One obvious problem is that we're hijacking an annotation to express something really different from what it was intended from. It would arguably be better to have a dedicated annotation for that kind of embedding, similar to the {{@IndexedEmbedded}} annotation from Hibernate Search.
But a less obvious and much more problematic consequence of that choice is that we are forced to put implicit limits on recursive models, and users cannot control these limits. Indeed, Infinispan 12 only uses static metamodels when defining Hibernate Search indexes, so infinite recursion is simply not an option. In the example above, we have to declare all fields at bootstrap, and that includes the recursions such as {{manager.id}}, {{manager.manager.id}}, {{manager.manager.manager.id}}, etc. Currently we limit embedding depth to {{7}}, which is very arbitrary.
To solve _that_ problem, we have two alternatives:
# Embrace infinite recursion, meaning we will define a dynamic schema. In the example above, we would define static fields {{manager}}, {{manager.id}} and {{manager.manager}}, and inside {{manager}} we would define dynamic fields for path patterns {{\*.manager}} (object field) and {{\*.manager.id}} (Integer field, stored, sortable). The main problem with this behavior is there is a possibility that user schemas introduce conflicts (e.g. two definitions of "manager" in two different embedded types), which will probably lead to runtime errors or non-matching queries.
# Switch to a dedicated annotation to configure embedding, e.g. {{@IndexedEmbedded}}. This annotation could be unlimited by default. Hibernate Search will throw an exception if that leads infinite recursion, and when that happens, users will be able to set attributes on the annotation to control recursion, for example with {{@IndexedEmbedded(includePaths = <finite list of field paths to embed>)}} or {{@IndexedEmbedded(includeDepth = <depth of recursion when indexing>)}}.
I'd be in favor of solution 2, which is the solution we implement in the Hibernate Search / Hibernate ORM mapper : https://docs.jboss.org/hibernate/search/6.0/reference/en-US/html_single/#...
Judging from a previous conversation, it seems [~anistor] [agrees|https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/...].
See also the full conversation, starting here: https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/topic/I...
> Expose indexed-embedding as a separate annotation in Protobuf index mapping
> ---------------------------------------------------------------------------
>
> Key: ISPN-12419
> URL: https://issues.redhat.com/browse/ISPN-12419
> Project: Infinispan
> Issue Type: Enhancement
> Components: Remote Querying
> Reporter: Yoann Rodière
> Priority: Major
> Labels: SearchNG
> Fix For: 12.0.0.Dev05
>
>
> This ticket is about a behavior change between Infinispan 11 and 12 caused by the upgrade to Hibernate Search 6; it must be addressed before the release of Infinispan 12.
> In Infinispan 11, when mapping a Protobuf type to an index, one must use the {{@Field}} annotation in order to mark a composite type as "embedded", e.g.:
> {code}
> message User {
> /**
> * @Field(store = Store.YES)
> * @SortableField
> */
> required int32 id = 1;
> /**
> * @Field(store = Store.YES)
> */
> optional User manager = 2;
> }
> {code}
> The {{@Field}} annotation on {{manager}} does not actually create a field: instead, it specifies that all fields defined in {{manager}} should be embedded in {{User}}. In this case, it means user will have a field named {{manager.id}}.
> One obvious problem is that we're hijacking an annotation to express something really different from what it was intended for. It would arguably be better to have a dedicated annotation for that kind of embedding, similar to the {{@IndexedEmbedded}} annotation from Hibernate Search.
> But a less obvious and much more problematic consequence of that choice is that we are forced to put implicit limits on recursive models, and users cannot control these limits. Indeed, Infinispan 12 only uses static metamodels when defining Hibernate Search indexes, so infinite recursion is simply not an option. In the example above, we have to declare all fields at bootstrap, and that includes the recursions such as {{manager.id}}, {{manager.manager.id}}, {{manager.manager.manager.id}}, etc. Currently we limit embedding depth to {{7}}, which is very arbitrary.
> To solve _that_ problem, we have two alternatives:
> # Embrace infinite recursion, meaning we will define a dynamic schema. In the example above, we would define static fields {{manager}}, {{manager.id}} and {{manager.manager}}, and inside {{manager}} we would define dynamic fields for path patterns {{\*.manager}} (object field) and {{\*.manager.id}} (Integer field, stored, sortable). The main problem with this behavior is there is a possibility that user schemas introduce conflicts (e.g. two definitions of "manager" in two different embedded types), which will probably lead to runtime errors or non-matching queries.
> # Switch to a dedicated annotation to configure embedding, e.g. {{@IndexedEmbedded}}. This annotation could be unlimited by default. Hibernate Search will throw an exception if that leads infinite recursion, and when that happens, users will be able to set attributes on the annotation to control recursion, for example with {{@IndexedEmbedded(includePaths = <finite list of field paths to embed>)}} or {{@IndexedEmbedded(includeDepth = <depth of recursion when indexing>)}}.
> I'd be in favor of solution 2, which is the solution we implement in the Hibernate Search / Hibernate ORM mapper : https://docs.jboss.org/hibernate/search/6.0/reference/en-US/html_single/#...
> Judging from a previous conversation, it seems [~anistor] [agrees|https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/...].
> See also the full conversation, starting here: https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/topic/I...
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12419) Expose indexed-embedding as a separate annotation in Protobuf index mapping
by Nistor Adrian (Jira)
[ https://issues.redhat.com/browse/ISPN-12419?page=com.atlassian.jira.plugi... ]
Nistor Adrian updated ISPN-12419:
---------------------------------
Labels: SearchNG (was: )
> Expose indexed-embedding as a separate annotation in Protobuf index mapping
> ---------------------------------------------------------------------------
>
> Key: ISPN-12419
> URL: https://issues.redhat.com/browse/ISPN-12419
> Project: Infinispan
> Issue Type: Enhancement
> Components: Remote Querying
> Reporter: Yoann Rodière
> Priority: Major
> Labels: SearchNG
> Fix For: 12.0.0.Dev05
>
>
> This ticket is about a behavior change between Infinispan 11 and 12 caused by the upgrade to Hibernate Search 6; it must be addressed before the release of Infinispan 12.
> In Infinispan 11, when mapping a Protobuf type to an index, one must use the {{@Field}} annotation in order to mark a composite type as "embedded", e.g.:
> {code}
> message User {
> /**
> * @Field(store = Store.YES)
> * @SortableField
> */
> required int32 id = 1;
> /**
> * @Field(store = Store.YES)
> */
> optional User manager = 2;
> }
> {code}
> The {{@Field}} annotation on {{manager}} does not actually create a field: instead, it specifies that all fields defined in {{manager}} should be embedded in {{User}}. In this case, it means user will have a field named {{manager.id}}.
> One obvious problem is that we're hijacking an annotation to express something really different from what it was intended from. It would arguably be better to have a dedicated annotation for that kind of embedding, similar to the {{@IndexedEmbedded}} annotation from Hibernate Search.
> But a less obvious and much more problematic consequence of that choice is that we are forced to put implicit limits on recursive models, and users cannot control these limits. Indeed, Infinispan 12 only uses static metamodels when defining Hibernate Search indexes, so infinite recursion is simply not an option. In the example above, we have to declare all fields at bootstrap, and that includes the recursions such as {{manager.id}}, {{manager.manager.id}}, {{manager.manager.manager.id}}, etc. Currently we limit embedding depth to {{7}}, which is very arbitrary.
> To solve _that_ problem, we have two alternatives:
> # Embrace infinite recursion, meaning we will define a dynamic schema. In the example above, we would define static fields {{manager}}, {{manager.id}} and {{manager.manager}}, and inside {{manager}} we would define dynamic fields for path patterns {{\*.manager}} (object field) and {{\*.manager.id}} (Integer field, stored, sortable). The main problem with this behavior is there is a possibility that user schemas introduce conflicts (e.g. two definitions of "manager" in two different embedded types), which will probably lead to runtime errors or non-matching queries.
> # Switch to a dedicated annotation to configure embedding, e.g. {{@IndexedEmbedded}}. This annotation could be unlimited by default. Hibernate Search will throw an exception if that leads infinite recursion, and when that happens, users will be able to set attributes on the annotation to control recursion, for example with {{@IndexedEmbedded(includePaths = <finite list of field paths to embed>)}} or {{@IndexedEmbedded(includeDepth = <depth of recursion when indexing>)}}.
> I'd be in favor of solution 2, which is the solution we implement in the Hibernate Search / Hibernate ORM mapper : https://docs.jboss.org/hibernate/search/6.0/reference/en-US/html_single/#...
> Judging from a previous conversation, it seems [~anistor] [agrees|https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/...].
> See also the full conversation, starting here: https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/topic/I...
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12419) Expose indexed-embedding as a separate annotation in Protobuf index mapping
by Nistor Adrian (Jira)
[ https://issues.redhat.com/browse/ISPN-12419?page=com.atlassian.jira.plugi... ]
Nistor Adrian updated ISPN-12419:
---------------------------------
Status: Open (was: New)
> Expose indexed-embedding as a separate annotation in Protobuf index mapping
> ---------------------------------------------------------------------------
>
> Key: ISPN-12419
> URL: https://issues.redhat.com/browse/ISPN-12419
> Project: Infinispan
> Issue Type: Enhancement
> Components: Remote Querying
> Reporter: Yoann Rodière
> Priority: Major
> Fix For: 12.0.0.Dev05
>
>
> This ticket is about a behavior change between Infinispan 11 and 12 caused by the upgrade to Hibernate Search 6; it must be addressed before the release of Infinispan 12.
> In Infinispan 11, when mapping a Protobuf type to an index, one must use the {{@Field}} annotation in order to mark a composite type as "embedded", e.g.:
> {code}
> message User {
> /**
> * @Field(store = Store.YES)
> * @SortableField
> */
> required int32 id = 1;
> /**
> * @Field(store = Store.YES)
> */
> optional User manager = 2;
> }
> {code}
> The {{@Field}} annotation on {{manager}} does not actually create a field: instead, it specifies that all fields defined in {{manager}} should be embedded in {{User}}. In this case, it means user will have a field named {{manager.id}}.
> One obvious problem is that we're hijacking an annotation to express something really different from what it was intended from. It would arguably be better to have a dedicated annotation for that kind of embedding, similar to the {{@IndexedEmbedded}} annotation from Hibernate Search.
> But a less obvious and much more problematic consequence of that choice is that we are forced to put implicit limits on recursive models, and users cannot control these limits. Indeed, Infinispan 12 only uses static metamodels when defining Hibernate Search indexes, so infinite recursion is simply not an option. In the example above, we have to declare all fields at bootstrap, and that includes the recursions such as {{manager.id}}, {{manager.manager.id}}, {{manager.manager.manager.id}}, etc. Currently we limit embedding depth to {{7}}, which is very arbitrary.
> To solve _that_ problem, we have two alternatives:
> # Embrace infinite recursion, meaning we will define a dynamic schema. In the example above, we would define static fields {{manager}}, {{manager.id}} and {{manager.manager}}, and inside {{manager}} we would define dynamic fields for path patterns {{\*.manager}} (object field) and {{\*.manager.id}} (Integer field, stored, sortable). The main problem with this behavior is there is a possibility that user schemas introduce conflicts (e.g. two definitions of "manager" in two different embedded types), which will probably lead to runtime errors or non-matching queries.
> # Switch to a dedicated annotation to configure embedding, e.g. {{@IndexedEmbedded}}. This annotation could be unlimited by default. Hibernate Search will throw an exception if that leads infinite recursion, and when that happens, users will be able to set attributes on the annotation to control recursion, for example with {{@IndexedEmbedded(includePaths = <finite list of field paths to embed>)}} or {{@IndexedEmbedded(includeDepth = <depth of recursion when indexing>)}}.
> I'd be in favor of solution 2, which is the solution we implement in the Hibernate Search / Hibernate ORM mapper : https://docs.jboss.org/hibernate/search/6.0/reference/en-US/html_single/#...
> Judging from a previous conversation, it seems [~anistor] [agrees|https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/...].
> See also the full conversation, starting here: https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/topic/I...
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12419) Expose indexed-embedding as a separate annotation in Protobuf index mapping
by Yoann Rodière (Jira)
Yoann Rodière created ISPN-12419:
------------------------------------
Summary: Expose indexed-embedding as a separate annotation in Protobuf index mapping
Key: ISPN-12419
URL: https://issues.redhat.com/browse/ISPN-12419
Project: Infinispan
Issue Type: Enhancement
Components: Remote Querying
Reporter: Yoann Rodière
Fix For: 12.0.0.Dev05
This ticket is about a behavior change between Infinispan 11 and 12 caused by the upgrade to Hibernate Search 6; it must be addressed before the release of Infinispan 12.
In Infinispan 11, when mapping a Protobuf type to an index, one must use the {{@Field}} annotation in order to mark a composite type as "embedded", e.g.:
{code}
message User {
/**
* @Field(store = Store.YES)
* @SortableField
*/
required int32 id = 1;
/**
* @Field(store = Store.YES)
*/
optional User manager = 2;
}
{code}
The {{@Field}} annotation on {{manager}} does not actually create a field: instead, it specifies that all fields defined in {{manager}} should be embedded in {{User}}. In this case, it means user will have a field named {{manager.id}}.
One obvious problem is that we're hijacking an annotation to express something really different from what it was intended from. It would arguably be better to have a dedicated annotation for that kind of embedding, similar to the {{@IndexedEmbedded}} annotation from Hibernate Search.
But a less obvious and much more problematic consequence of that choice is that we are forced to put implicit limits on recursive models, and users cannot control these limits. Indeed, Infinispan 12 only uses static metamodels when defining Hibernate Search indexes, so infinite recursion is simply not an option. In the example above, we have to declare all fields at bootstrap, and that includes the recursions such as {{manager.id}}, {{manager.manager.id}}, {{manager.manager.manager.id}}, etc. Currently we limit embedding depth to {{7}}, which is very arbitrary.
To solve _that_ problem, we have two alternatives:
# Embrace infinite recursion, meaning we will define a dynamic schema. In the example above, we would define static fields {{manager}}, {{manager.id}} and {{manager.manager}}, and inside {{manager}} we would define dynamic fields for path patterns {{\*.manager}} (object field) and {{\*.manager.id}} (Integer field, stored, sortable). The main problem with this behavior is there is a possibility that user schemas introduce conflicts (e.g. two definitions of "manager" in two different embedded types), which will probably lead to runtime errors or non-matching queries.
# Switch to a dedicated annotation to configure embedding, e.g. {{@IndexedEmbedded}}. This annotation could be unlimited by default. Hibernate Search will throw an exception if that leads infinite recursion, and when that happens, users will be able to set attributes on the annotation to control recursion, for example with {{@IndexedEmbedded(includePaths = <finite list of field paths to embed>)}} or {{@IndexedEmbedded(includeDepth = <depth of recursion when indexing>)}}.
I'd be in favor of solution 2, which is the solution we implement in the Hibernate Search / Hibernate ORM mapper : https://docs.jboss.org/hibernate/search/6.0/reference/en-US/html_single/#...
Judging from a previous conversation, it seems [~anistor] [agrees|https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/...].
See also the full conversation, starting here: https://infinispan.zulipchat.com/#narrow/stream/118645-infinispan/topic/I...
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12418) Docs: Clustered Locks
by Donald Naro (Jira)
Donald Naro created ISPN-12418:
----------------------------------
Summary: Docs: Clustered Locks
Key: ISPN-12418
URL: https://issues.redhat.com/browse/ISPN-12418
Project: Infinispan
Issue Type: Enhancement
Components: Documentation
Affects Versions: 12.0.0.Dev04, 11.0.4.Final
Reporter: Donald Naro
Assignee: Donald Naro
Need to update Clustered Locks docu. Currently exposes details for functionality that is not implemented and is labelled as experimental.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months
[Red Hat JIRA] (ISPN-12401) Support for query and indexing statistics
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-12401?page=com.atlassian.jira.plugi... ]
Work on ISPN-12401 started by Gustavo Fernandes.
------------------------------------------------
> Support for query and indexing statistics
> -----------------------------------------
>
> Key: ISPN-12401
> URL: https://issues.redhat.com/browse/ISPN-12401
> Project: Infinispan
> Issue Type: Enhancement
> Components: Embedded Querying
> Affects Versions: 12.0.0.Dev04
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Priority: Major
>
> After the migration to Hibernate Search 6, the statistics are not exposed. In Infinispan 11, stats were only available for indexed queries and not hybrid or non-indexed queries, apart from being scoped at a single cache manager level and not cluster level.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
3 years, 6 months