[JBoss JIRA] (ISPN-12186) Upgrade to Hibernate Search 6.0.0.Beta9
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-12186?page=com.atlassian.jira.plugi... ]
Gustavo Fernandes updated ISPN-12186:
-------------------------------------
Status: Pull Request Sent (was: Coding In Progress)
Git Pull Request: https://github.com/infinispan/infinispan/pull/8612
> Upgrade to Hibernate Search 6.0.0.Beta9
> ---------------------------------------
>
> Key: ISPN-12186
> URL: https://issues.redhat.com/browse/ISPN-12186
> Project: Infinispan
> Issue Type: Task
> Components: Embedded Querying, Remote Querying
> Reporter: Yoann Rodière
> Assignee: Gustavo Fernandes
> Priority: Major
> Fix For: 12.0.0.Dev02
>
>
> TODO:
> * Upgrade the Hibernate Search dependency (obviously); be warned the Lucene dependency is now 8.6.0
> * Handle [deprecations and breaking changes|https://in.relation.to/2020/08/03/hibernate-search-6-0-0-Beta9/#breaking_changes]
> * Simplify the configuration properties generated by Infinispan and passed to Hibernate Search; see [here|https://in.relation.to/2020/08/03/hibernate-search-6-0-0-Beta9/#simpler-configuration-scheme].
> * Simplify the generation of IDs: {{org.hibernate.search.mapper.pojo.work.spi.PojoIndexer}} now allows passing the routing key explicitly (in our case, {{String.valueOf(segmentId)}}), so our custom implementation of {{RoutingKeyBridge}} (which relies on a hack) is no longer necessary.
> * As a result of the previous item, we can also get rid of the hack that concatenates the segment ID to the document ID; see ISPN-12170.
>
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12203) @SortableField annotation is ignored in nested message
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-12203?page=com.atlassian.jira.plugi... ]
Gustavo Fernandes updated ISPN-12203:
-------------------------------------
Summary: @SortableField annotation is ignored in nested message (was: @SortableField annotation is ignored in sub-message)
> @SortableField annotation is ignored in nested message
> ------------------------------------------------------
>
> Key: ISPN-12203
> URL: https://issues.redhat.com/browse/ISPN-12203
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying, Remote Querying
> Affects Versions: 11.0.3.Final
> Reporter: Gustavo Fernandes
> Priority: Major
>
> Consider the protofile:
> {code:java}
> /* @Indexed */
> message ProcessInstance {
>
> /* @Field(store = Store.YES) */
> optional ProcessInstanceError error = 14;
> }
> /* @Indexed */
> message ProcessInstanceError {
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string nodeDefinitionId = 1;
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string message = 2;
> }
> {code}
> The sortable annotations in the subtype are not taken into consideration and the query fails with:
> {noformat}
> SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured
> for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index
> processinstances_protobuf, thus an uninverting reader must be created. You should declare the
> missing sort fields using @SortableField.
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12203) @SortableField annotation is ignored in sub-message
by Nistor Adrian (Jira)
[ https://issues.redhat.com/browse/ISPN-12203?page=com.atlassian.jira.plugi... ]
Nistor Adrian updated ISPN-12203:
---------------------------------
Status: Open (was: New)
> @SortableField annotation is ignored in sub-message
> ---------------------------------------------------
>
> Key: ISPN-12203
> URL: https://issues.redhat.com/browse/ISPN-12203
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying, Remote Querying
> Affects Versions: 11.0.3.Final
> Reporter: Gustavo Fernandes
> Priority: Major
>
> Consider the protofile:
> {code:java}
> /* @Indexed */
> message ProcessInstance {
>
> /* @Field(store = Store.YES) */
> optional ProcessInstanceError error = 14;
> }
> /* @Indexed */
> message ProcessInstanceError {
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string nodeDefinitionId = 1;
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string message = 2;
> }
> {code}
> The sortable annotations in the subtype are not taken into consideration and the query fails with:
> {noformat}
> SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured
> for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index
> processinstances_protobuf, thus an uninverting reader must be created. You should declare the
> missing sort fields using @SortableField.
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12203) @SortableField annotation is ignored in sub-message
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-12203?page=com.atlassian.jira.plugi... ]
Gustavo Fernandes updated ISPN-12203:
-------------------------------------
Summary: @SortableField annotation is ignored in sub-message (was: @SortableField annotation is ignored in sub-type)
> @SortableField annotation is ignored in sub-message
> ---------------------------------------------------
>
> Key: ISPN-12203
> URL: https://issues.redhat.com/browse/ISPN-12203
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying, Remote Querying
> Affects Versions: 11.0.3.Final
> Reporter: Gustavo Fernandes
> Priority: Major
>
> Consider the protofile:
> {code:java}
> /* @Indexed */
> message ProcessInstance {
>
> /* @Field(store = Store.YES) */
> optional ProcessInstanceError error = 14;
> }
> /* @Indexed */
> message ProcessInstanceError {
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string nodeDefinitionId = 1;
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string message = 2;
> }
> {code}
> The sortable annotations in the subtype are not taken into consideration and the query fails with:
> {noformat}
> SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured
> for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index
> processinstances_protobuf, thus an uninverting reader must be created. You should declare the
> missing sort fields using @SortableField.
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12203) @SortableField annotation is ignored in sub-type
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-12203?page=com.atlassian.jira.plugi... ]
Gustavo Fernandes updated ISPN-12203:
-------------------------------------
Description:
Consider the protofile:
{code:java}
/* @Indexed */
message ProcessInstance {
/* @Field(store = Store.YES) */
optional ProcessInstanceError error = 14;
}
/* @Indexed */
message ProcessInstanceError {
/* @Field(store = Store.YES)
* @SortableField
*/
optional string nodeDefinitionId = 1;
/* @Field(store = Store.YES)
* @SortableField
*/
optional string message = 2;
}
{code}
The sortable annotations in the subtype are not taken into consideration and the query fails with:
{noformat}
SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured
for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index
processinstances_protobuf, thus an uninverting reader must be created. You should declare the
missing sort fields using @SortableField.
{noformat}
was:
Consider the protofile:
{code:java}
/* @Indexed */
message ProcessInstance {
/* @Field(store = Store.YES) */
optional ProcessInstanceError error = 14;
}
/* @Indexed */
message ProcessInstanceError {
/* @Field(store = Store.YES)
* @SortableField
*/
optional string nodeDefinitionId = 1;
/* @Field(store = Store.YES)
* @SortableField
*/
optional string message = 2;
}
{code}
The sortable annotations in the subtype are not taken into consideration and the query fails with:
{noformat}
SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index processinstances_protobuf, thus
an uninverting reader must be created. You should declare the missing sort fields using @SortableField.
{noformat}
> @SortableField annotation is ignored in sub-type
> ------------------------------------------------
>
> Key: ISPN-12203
> URL: https://issues.redhat.com/browse/ISPN-12203
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying, Remote Querying
> Affects Versions: 11.0.3.Final
> Reporter: Gustavo Fernandes
> Priority: Major
>
> Consider the protofile:
> {code:java}
> /* @Indexed */
> message ProcessInstance {
>
> /* @Field(store = Store.YES) */
> optional ProcessInstanceError error = 14;
> }
> /* @Indexed */
> message ProcessInstanceError {
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string nodeDefinitionId = 1;
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string message = 2;
> }
> {code}
> The sortable annotations in the subtype are not taken into consideration and the query fails with:
> {noformat}
> SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured
> for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index
> processinstances_protobuf, thus an uninverting reader must be created. You should declare the
> missing sort fields using @SortableField.
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12203) @SortableField annotation is ignored in sub-type
by Gustavo Fernandes (Jira)
[ https://issues.redhat.com/browse/ISPN-12203?page=com.atlassian.jira.plugi... ]
Gustavo Fernandes updated ISPN-12203:
-------------------------------------
Description:
Consider the protofile:
{code:java}
/* @Indexed */
message ProcessInstance {
/* @Field(store = Store.YES) */
optional ProcessInstanceError error = 14;
}
/* @Indexed */
message ProcessInstanceError {
/* @Field(store = Store.YES)
* @SortableField
*/
optional string nodeDefinitionId = 1;
/* @Field(store = Store.YES)
* @SortableField
*/
optional string message = 2;
}
{code}
The sortable annotations in the subtype are not taken into consideration and the query fails with:
{noformat}
SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index processinstances_protobuf, thus
an uninverting reader must be created. You should declare the missing sort fields using @SortableField.
{noformat}
was:
Consider the protofile:
{code}
/* @Indexed */
message ProcessInstance {
/* @Field(store = Store.YES) */
optional ProcessInstanceError error = 14;
}
/* @Indexed */
message ProcessInstanceError {
/* @Field(store = Store.YES)
* @SortableField
*/
optional string nodeDefinitionId = 1;
/* @Field(store = Store.YES)
* @SortableField
*/
optional string message = 2;
}
{code}
The sortable annotations in the subtype are not taken into consideration and the query fails with:
{noformat}
SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index processinstances_protobuf, thus an uninverting reader must be created. You should declare the missing sort fields using @SortableField.
{noformat}
> @SortableField annotation is ignored in sub-type
> ------------------------------------------------
>
> Key: ISPN-12203
> URL: https://issues.redhat.com/browse/ISPN-12203
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying, Remote Querying
> Affects Versions: 11.0.3.Final
> Reporter: Gustavo Fernandes
> Priority: Major
>
> Consider the protofile:
> {code:java}
> /* @Indexed */
> message ProcessInstance {
>
> /* @Field(store = Store.YES) */
> optional ProcessInstanceError error = 14;
> }
> /* @Indexed */
> message ProcessInstanceError {
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string nodeDefinitionId = 1;
> /* @Field(store = Store.YES)
> * @SortableField
> */
> optional string message = 2;
> }
> {code}
> The sortable annotations in the subtype are not taken into consideration and the query fails with:
> {noformat}
> SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index processinstances_protobuf, thus
> an uninverting reader must be created. You should declare the missing sort fields using @SortableField.
> {noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12203) @SortableField annotation is ignored in sub-type
by Gustavo Fernandes (Jira)
Gustavo Fernandes created ISPN-12203:
----------------------------------------
Summary: @SortableField annotation is ignored in sub-type
Key: ISPN-12203
URL: https://issues.redhat.com/browse/ISPN-12203
Project: Infinispan
Issue Type: Bug
Components: Embedded Querying, Remote Querying
Affects Versions: 11.0.3.Final
Reporter: Gustavo Fernandes
Consider the protofile:
{code}
/* @Indexed */
message ProcessInstance {
/* @Field(store = Store.YES) */
optional ProcessInstanceError error = 14;
}
/* @Indexed */
message ProcessInstanceError {
/* @Field(store = Store.YES)
* @SortableField
*/
optional string nodeDefinitionId = 1;
/* @Field(store = Store.YES)
* @SortableField
*/
optional string message = 2;
}
{code}
The sortable annotations in the subtype are not taken into consideration and the query fails with:
{noformat}
SearchException: HSEARCH000301: Requested sort field(s) error.nodeDefinitionId are not configured for entity type org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper mapped to index processinstances_protobuf, thus an uninverting reader must be created. You should declare the missing sort fields using @SortableField.
{noformat}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-11176) XSite Max Idle
by Paul Ferraro (Jira)
[ https://issues.redhat.com/browse/ISPN-11176?page=com.atlassian.jira.plugi... ]
Paul Ferraro commented on ISPN-11176:
-------------------------------------
At the last F2F, while discussing locking w/x-site, I brought up the possibility of partitioning entries with a primary site (in the similar way that we partition them within a single cluster). With this in place, max-idle processing would be initiated only on the primary site for a given entry and cascaded to backup sites accordingly. It would, however, require that reads from the backup sites "touch" the primary site. This strategy has the advantage of minimal x-site traffic (for non-exceptional cases) and simplifies max-idle processing by using a single authority for determining when to expire a given entry thereby preventing premature expirations.
In the absence of this, however, option 1 seems like the better solution.
> XSite Max Idle
> --------------
>
> Key: ISPN-11176
> URL: https://issues.redhat.com/browse/ISPN-11176
> Project: Infinispan
> Issue Type: Enhancement
> Components: Cross-Site Replication, Expiration
> Reporter: Will Burns
> Assignee: Will Burns
> Priority: Major
> Fix For: 12.0.0.Final
>
>
> Max idle expiration currently doesn't work with xsite. That is if an entry was written and replicated to both sites but one site never reads the value, but the other does. If they then need to read the value from the other site it will be expired (assuming the max idle time has elapsed).
> There are a few ways we can do this.
> 1. Keep access times local to every site. When a site finds an entry is expired it asks the other site(s) if it has a more recent access. If a site is known to have gone down we should touch all entries, since they may not have updated access times. Requires very little additional xsite communication.
> 2. Batch touch commands and only send every so often. Has window of loss, but should be small. Requires more site usage. Wouldn't work for really low max idle times as an entry could expire before the touch command is replicated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-11176) XSite Max Idle
by Will Burns (Jira)
[ https://issues.redhat.com/browse/ISPN-11176?page=com.atlassian.jira.plugi... ]
Will Burns commented on ISPN-11176:
-----------------------------------
{quote}I was asking mostly because I'm trying to understand if we could maybe use a similar strategy to avoid the sync touch commands for in-cluster max-idle.
{quote}
{quote}Yeah, sorry. I meant to ask about the touch command in the remote site.
{quote}
Combining these two. The problem if it is async is you have a window of consistency loss if a node is taken down. We can mitigate this issue by performing a touch all when it occurs. I was told this is not acceptable, and I agree, for clustered max idle where it would occur for a single node. However, I believe that it should be okay when it would occur only when an entire site is lost.
Thinking further when we have another site all touch commands can be async as we have the other site to protect us and at worst if the other site goes down it will be refreshed anyways. I will try to do this.
{quote}I think it would be "more consistent" if we sent the local version in the "check last version" command to Site 2, and it would reply with "expired" so the local site would treat the entry as expired.
{quote}
I like this idea, I will have to tweak how our existing TouchCommand works to support the version check.
> XSite Max Idle
> --------------
>
> Key: ISPN-11176
> URL: https://issues.redhat.com/browse/ISPN-11176
> Project: Infinispan
> Issue Type: Enhancement
> Components: Cross-Site Replication, Expiration
> Reporter: Will Burns
> Assignee: Will Burns
> Priority: Major
> Fix For: 12.0.0.Final
>
>
> Max idle expiration currently doesn't work with xsite. That is if an entry was written and replicated to both sites but one site never reads the value, but the other does. If they then need to read the value from the other site it will be expired (assuming the max idle time has elapsed).
> There are a few ways we can do this.
> 1. Keep access times local to every site. When a site finds an entry is expired it asks the other site(s) if it has a more recent access. If a site is known to have gone down we should touch all entries, since they may not have updated access times. Requires very little additional xsite communication.
> 2. Batch touch commands and only send every so often. Has window of loss, but should be small. Requires more site usage. Wouldn't work for really low max idle times as an entry could expire before the touch command is replicated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-11176) XSite Max Idle
by Dan Berindei (Jira)
[ https://issues.redhat.com/browse/ISPN-11176?page=com.atlassian.jira.plugi... ]
Dan Berindei commented on ISPN-11176:
-------------------------------------
{quote}I have changed gears so that the system clock time does not really matter as everything should just be a replicated touch. There is no reason I can think of currently as to why we need an offset value. This makes time a lot easier as we don't have to worry about clock skews.
{quote}
(y)
{quote}Even if the remote site node that sends the async touch crashes at worst it means we could have another xsite max idle check on the remote site asking the originator site. Consistency should be fine, just possibly increased latency for a given request.
{quote}
(y)
{quote}I kept it as sync to be inline with clustered max idle where it is also sync. It being async without changing part of xsite max idle would cause some consistency issues. But this may need to be changed per other comments.
{quote}
Fair enough. When we already have one x-site RPC, one more in-cluster RPC is not going to add much to the latency of the operation anyway.
I was asking mostly because I'm trying to understand if we could maybe use a similar strategy to avoid the sync touch commands for in-cluster max-idle.
{quote}This is the same as the prior comment?
{quote}
Yeah, sorry. I meant to ask about the touch command in the remote site.
{quote}I think I should have been more specific as to what I meant by "an issue". In this case the backup and primary will both send an xsite touch command, instead of just 1.
{quote}
That doesn't sound like an issue to me, just a trade-off between having one extra in-cluster RPC (adding to the latency) and one parallel x-site RPC.
I was reading it as "we may have an issue [...] possibly losing the write", that sounded a lot scarier :)
{quote}Yes, getting the previous value would require for a write of any expired entry to also do a touch or get previous value from the other site. This seems like a lot of overhead and can be documented.
Also xsite can cause previous value to not be correct if the other site has a write that has not yet been replicated. So I personally feel like the expiration not returning the correct one should be okay.
{quote}
Sure, IRAC is going to allow commands to see a value that was already modified on the remote site, but that is only going to happen while IRAC hasn't replicated the update yet.
With expiration I believe it's possible for an entry to expire at time T, then a read at T+10m tries in Site 1 finds the old value hasn't expired yet because Site 2 has updated it at T+9.99m. Although 1m is probably a more realistic example, as the purge task is also going to remove expired entries entries every 1m (unless it can't keep up?)
I think it would be "more consistent" if we sent the local version in the "check last version" command to Site 2, and it would reply with "expired" so the local site would treat the entry as expired.
TBH if we sent the local version with the "check last version" command, maybe it wouldn't even be such a big overhead to send the value back in the response. But it seems pointless knowing that updates of non-expired are not replicated this way.
{quote}This one was in reference to listeners (which is all above). IRAC with conflict of writes will miss previous value for different writes as it wasn't replicated to the other site.
{quote}
I think that's fine, listeners are going to see a miss the previous value only if they weren't notified for that value, so the history they see is consistent.
All it means is that listeners can't expect the sequence of notifications to match the sequence of invoked cache operations, and because of topology retries that applies even without x-site.
{quote}Yeah, I need to think through it a bit more, but I believe we need to do the touch command only on primary and to fail the command as necessary so it isn't written to primary or backup.
{quote}
(y)
{quote}Agreed, will have to think through those as well, but just took a first stab when topology is stable at least.
{quote}
(y)
> XSite Max Idle
> --------------
>
> Key: ISPN-11176
> URL: https://issues.redhat.com/browse/ISPN-11176
> Project: Infinispan
> Issue Type: Enhancement
> Components: Cross-Site Replication, Expiration
> Reporter: Will Burns
> Assignee: Will Burns
> Priority: Major
> Fix For: 12.0.0.Final
>
>
> Max idle expiration currently doesn't work with xsite. That is if an entry was written and replicated to both sites but one site never reads the value, but the other does. If they then need to read the value from the other site it will be expired (assuming the max idle time has elapsed).
> There are a few ways we can do this.
> 1. Keep access times local to every site. When a site finds an entry is expired it asks the other site(s) if it has a more recent access. If a site is known to have gone down we should touch all entries, since they may not have updated access times. Requires very little additional xsite communication.
> 2. Batch touch commands and only send every so often. Has window of loss, but should be small. Requires more site usage. Wouldn't work for really low max idle times as an entry could expire before the touch command is replicated.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months