[JBoss JIRA] (ISPN-12171) Set store segmented attribute dynamically based upon store characteristics
by Will Burns (Jira)
[ https://issues.redhat.com/browse/ISPN-12171?page=com.atlassian.jira.plugi... ]
Will Burns updated ISPN-12171:
------------------------------
Status: Open (was: New)
> Set store segmented attribute dynamically based upon store characteristics
> --------------------------------------------------------------------------
>
> Key: ISPN-12171
> URL: https://issues.redhat.com/browse/ISPN-12171
> Project: Infinispan
> Issue Type: Enhancement
> Components: Loaders and Stores
> Reporter: Will Burns
> Priority: Major
> Fix For: 12.0.0.Final
>
>
> We now have the new Store SPI which has the `characteristics` method which allows a store to publish what operations/configurations it supports. We should utilize this to determine if a store is segmented or not when the user doesn't specify the configuration value.
> If a Store can dynamically change supporting the segmentation characteristic it should log a message or possibly error if needed.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12171) Set store segmented attribute dynamically based upon store characteristics
by Will Burns (Jira)
[ https://issues.redhat.com/browse/ISPN-12171?page=com.atlassian.jira.plugi... ]
Work on ISPN-12171 started by Will Burns.
-----------------------------------------
> Set store segmented attribute dynamically based upon store characteristics
> --------------------------------------------------------------------------
>
> Key: ISPN-12171
> URL: https://issues.redhat.com/browse/ISPN-12171
> Project: Infinispan
> Issue Type: Enhancement
> Components: Loaders and Stores
> Reporter: Will Burns
> Assignee: Will Burns
> Priority: Major
> Fix For: 12.0.0.Final
>
>
> We now have the new Store SPI which has the `characteristics` method which allows a store to publish what operations/configurations it supports. We should utilize this to determine if a store is segmented or not when the user doesn't specify the configuration value.
> If a Store can dynamically change supporting the segmentation characteristic it should log a message or possibly error if needed.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12172) Make the number of segments dynamic
by Dan Berindei (Jira)
Dan Berindei created ISPN-12172:
-----------------------------------
Summary: Make the number of segments dynamic
Key: ISPN-12172
URL: https://issues.redhat.com/browse/ISPN-12172
Project: Infinispan
Issue Type: Bug
Components: Configuration, Core
Affects Versions: 11.0.1.Final
Reporter: Dan Berindei
Assignee: Dan Berindei
Fix For: 12.0.0.Final
Now that the data container and stores store segment information, it's becoming harder and harder to change the number of segments to accommodate a growth in the number of nodes and preserve all the data.
We should investigate removing the {{numSegments}} configuration attribute and using an {{effectiveNumSegments}} that is the first power of 2 that's bigger than 10 * <cluster size> (customizable, maybe?) and at least as big as the previous {{effectiveNumSegments}}.
The number of segments increase would be decided by the coordinator on join and would require state transfer, as {{SyncConsistentHashFactory}} would not assign the split segments to exactly the same owners (although it would have to be pretty close in order to make this practical).
We would like to avoid updating the segment of every persisted entry on {{effectiveNumSegments}} changes, so the segment stored in persisted entries should be computed based on the maximum value we will allow for {{effectiveNumSegments}}, e.g. 2^16. We would then use a shift+mask to obtain the "effective segment" of the key, and range queries when obtaining all the keys in a segment.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12171) Set store segmented attribute dynamically based upon store characteristics
by Will Burns (Jira)
Will Burns created ISPN-12171:
---------------------------------
Summary: Set store segmented attribute dynamically based upon store characteristics
Key: ISPN-12171
URL: https://issues.redhat.com/browse/ISPN-12171
Project: Infinispan
Issue Type: Enhancement
Components: Loaders and Stores
Reporter: Will Burns
Fix For: 12.0.0.Final
We now have the new Store SPI which has the `characteristics` method which allows a store to publish what operations/configurations it supports. We should utilize this to determine if a store is segmented or not when the user doesn't specify the configuration value.
If a Store can dynamically change supporting the segmentation characteristic it should log a message or possibly error if needed.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12170) Query - Stop encoding the segment ID in document IDs
by Yoann Rodière (Jira)
[ https://issues.redhat.com/browse/ISPN-12170?page=com.atlassian.jira.plugi... ]
Yoann Rodière updated ISPN-12170:
---------------------------------
Summary: Query - Stop encoding the segment ID in document IDs (was: Query - Stop encoding the routing key in document IDs)
> Query - Stop encoding the segment ID in document IDs
> ----------------------------------------------------
>
> Key: ISPN-12170
> URL: https://issues.redhat.com/browse/ISPN-12170
> Project: Infinispan
> Issue Type: Task
> Components: Embedded Querying, Remote Querying
> Affects Versions: 12.0.0.Dev01
> Reporter: Yoann Rodière
> Priority: Major
> Fix For: 12.0.0.Dev02
>
>
> We're currently relying on a hack to determine routing keys for indexed documents in Infinispan Query. That is, we concatenate the ID with the segment ID, and use a routing key bridge to extract the segment ID and use it as a routing key.
>
> A nasty side-effect is that we include the segment ID in the ID of indexed documents in the index.
>
> With Hibernate Search 6.0.0.Beta9, this won't be necessary anymore: we'll be able to pass the segment ID (routing key) to Hibernate Search directly. This means we'll be able to remove the hack and to use cleaner document IDs.
>
> This removal means documents will have a different ID, and thus any existing index will have to be rebuilt. Thus this should be done ASAP.
>
> See the implementation of RoutingKeyBridge in Infinispan, and see https://hibernate.atlassian.net/browse/HSEARCH-3891
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12170) Query - Stop encoding the segment ID in document IDs
by Yoann Rodière (Jira)
[ https://issues.redhat.com/browse/ISPN-12170?page=com.atlassian.jira.plugi... ]
Yoann Rodière updated ISPN-12170:
---------------------------------
Description:
We're currently relying on a hack to determine routing keys for indexed documents in Infinispan Query. That is, we concatenate the ID with the segment ID, and use a routing key bridge to extract the segment ID then use it as a routing key.
A nasty side-effect is that we include the segment ID in the ID of indexed documents in the index.
With Hibernate Search 6.0.0.Beta9, this won't be necessary anymore: we'll be able to pass the segment ID (routing key) to Hibernate Search directly. This means we'll be able to remove the hack and to use cleaner document IDs.
This removal means documents will have a different ID, and thus any existing index will have to be rebuilt. Thus this should be done ASAP.
See the implementation of RoutingKeyBridge in Infinispan, and see [https://hibernate.atlassian.net/browse/HSEARCH-3891]
was:
We're currently relying on a hack to determine routing keys for indexed documents in Infinispan Query. That is, we concatenate the ID with the segment ID, and use a routing key bridge to extract the segment ID and use it as a routing key.
A nasty side-effect is that we include the segment ID in the ID of indexed documents in the index.
With Hibernate Search 6.0.0.Beta9, this won't be necessary anymore: we'll be able to pass the segment ID (routing key) to Hibernate Search directly. This means we'll be able to remove the hack and to use cleaner document IDs.
This removal means documents will have a different ID, and thus any existing index will have to be rebuilt. Thus this should be done ASAP.
See the implementation of RoutingKeyBridge in Infinispan, and see https://hibernate.atlassian.net/browse/HSEARCH-3891
> Query - Stop encoding the segment ID in document IDs
> ----------------------------------------------------
>
> Key: ISPN-12170
> URL: https://issues.redhat.com/browse/ISPN-12170
> Project: Infinispan
> Issue Type: Task
> Components: Embedded Querying, Remote Querying
> Affects Versions: 12.0.0.Dev01
> Reporter: Yoann Rodière
> Priority: Major
> Fix For: 12.0.0.Dev02
>
>
> We're currently relying on a hack to determine routing keys for indexed documents in Infinispan Query. That is, we concatenate the ID with the segment ID, and use a routing key bridge to extract the segment ID then use it as a routing key.
>
> A nasty side-effect is that we include the segment ID in the ID of indexed documents in the index.
>
> With Hibernate Search 6.0.0.Beta9, this won't be necessary anymore: we'll be able to pass the segment ID (routing key) to Hibernate Search directly. This means we'll be able to remove the hack and to use cleaner document IDs.
>
> This removal means documents will have a different ID, and thus any existing index will have to be rebuilt. Thus this should be done ASAP.
>
> See the implementation of RoutingKeyBridge in Infinispan, and see [https://hibernate.atlassian.net/browse/HSEARCH-3891]
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months
[JBoss JIRA] (ISPN-12170) Query - Stop encoding the routing key in document IDs
by Yoann Rodière (Jira)
Yoann Rodière created ISPN-12170:
------------------------------------
Summary: Query - Stop encoding the routing key in document IDs
Key: ISPN-12170
URL: https://issues.redhat.com/browse/ISPN-12170
Project: Infinispan
Issue Type: Task
Components: Embedded Querying, Remote Querying
Affects Versions: 12.0.0.Dev01
Reporter: Yoann Rodière
Fix For: 12.0.0.Dev02
We're currently relying on a hack to determine routing keys for indexed documents in Infinispan Query. That is, we concatenate the ID with the segment ID, and use a routing key bridge to extract the segment ID and use it as a routing key.
A nasty side-effect is that we include the segment ID in the ID of indexed documents in the index.
With Hibernate Search 6.0.0.Beta9, this won't be necessary anymore: we'll be able to pass the segment ID (routing key) to Hibernate Search directly. This means we'll be able to remove the hack and to use cleaner document IDs.
This removal means documents will have a different ID, and thus any existing index will have to be rebuilt. Thus this should be done ASAP.
See the implementation of RoutingKeyBridge in Infinispan, and see https://hibernate.atlassian.net/browse/HSEARCH-3891
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
5 years, 8 months