[JBoss JIRA] (ISPN-5451) Data Container Segment Striping
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-5451?page=com.atlassian.jira.plugin.... ]
William Burns updated ISPN-5451:
--------------------------------
Description:
Today each node owns a given number of segments at a time. There are many operations that are only concerned with entries of a given segment. Unfortunately this means that we must iterate over the entire contents of the cache to find only the entries that match given segments. Note this JIRA is only for the in memory data container. The cache store requires additional work and should be done separately as it may involve reworking many more areas of the code base.
# State Transfer entry removal (this removes all entries of given segments)
# Entry retrieval (if we could iterate by segment instead of over all values we could complete segments sooner which would reduce how much memory a client must hold)
# Indexing may want to index by segment at some point as well
was:
Today each node owns a given number of segments at a time. There are many operations that are only concerned with entries of a given segment. Unfortunately this means that we must iterate over the entire contents of the cache to find only the entries that match given segments. Note this JIRA is only for the in memory data container. The cache store requires additional work and should be done separately as it may involve reworking many more areas of the code base.
# State Transfer entry removal (this removes all entries of given segments)
# Entry retrieval (if we could iterate by segment instead of over all values we could complete segments sooner which would reduce how much memory a client must hold)
> Data Container Segment Striping
> -------------------------------
>
> Key: ISPN-5451
> URL: https://issues.jboss.org/browse/ISPN-5451
> Project: Infinispan
> Issue Type: Feature Request
> Components: Core
> Reporter: William Burns
> Assignee: William Burns
> Fix For: 8.1.0.Final
>
>
> Today each node owns a given number of segments at a time. There are many operations that are only concerned with entries of a given segment. Unfortunately this means that we must iterate over the entire contents of the cache to find only the entries that match given segments. Note this JIRA is only for the in memory data container. The cache store requires additional work and should be done separately as it may involve reworking many more areas of the code base.
> # State Transfer entry removal (this removes all entries of given segments)
> # Entry retrieval (if we could iterate by segment instead of over all values we could complete segments sooner which would reduce how much memory a client must hold)
> # Indexing may want to index by segment at some point as well
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5451) Data Container Segment Striping
by William Burns (JIRA)
William Burns created ISPN-5451:
-----------------------------------
Summary: Data Container Segment Striping
Key: ISPN-5451
URL: https://issues.jboss.org/browse/ISPN-5451
Project: Infinispan
Issue Type: Feature Request
Components: Core
Reporter: William Burns
Assignee: William Burns
Fix For: 8.1.0.Final
Today each node owns a given number of segments at a time. There are many operations that are only concerned with entries of a given segment. Unfortunately this means that we must iterate over the entire contents of the cache to find only the entries that match given segments. Note this JIRA is only for the in memory data container. The cache store requires additional work and should be done separately as it may involve reworking many more areas of the code base.
# State Transfer entry removal (this removes all entries of given segments)
# Entry retrieval (if we could iterate by segment instead of over all values we could complete segments sooner which would reduce how much memory a client must hold)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-3985) In BCHM traverse internal segments for parallel map/reduce execution
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-3985?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-3985:
-------------------------------------
This appears to no longer be required with https://issues.jboss.org/browse/ISPN-3023 the new BoundedConcurrentHashMap implementation supports the proper forEach method that uses the fork join task stealing algorithms. Although we may want to remove the now unneeded ParallelIterableMap interface.
> In BCHM traverse internal segments for parallel map/reduce execution
> --------------------------------------------------------------------
>
> Key: ISPN-3985
> URL: https://issues.jboss.org/browse/ISPN-3985
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core
> Affects Versions: 7.0.0.Final
> Reporter: Vladimir Blagojevic
> Fix For: 8.0.0.Final
>
>
> Currently when BoundedConcurrentHashMap is used in DataContainer we split input keys and traverse key/value pairs in parallel using executor. That is all good, however, we should optimize this solution as each segment in BCHM is a separate map we can iterate over each segment in a separate thread rather than blindly splitting input keys.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-2041) Provide putAll(map) and getAll(map) bulk operation on Cache and RemoteCache
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-2041?page=com.atlassian.jira.plugin.... ]
William Burns resolved ISPN-2041.
---------------------------------
Fix Version/s: (was: 8.0.0.Final)
Resolution: Duplicate Issue
This was fixed in ISPN-5265 & ISPN-5266
> Provide putAll(map) and getAll(map) bulk operation on Cache and RemoteCache
> ---------------------------------------------------------------------------
>
> Key: ISPN-2041
> URL: https://issues.jboss.org/browse/ISPN-2041
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core
> Affects Versions: 5.1.4.FINAL
> Reporter: susanin
> Assignee: William Burns
> Priority: Minor
>
> Currently Infinispan implements putAll(map) by means of looping over all entries and submitting them by means of put() one-by-one.
> This is very slow if you have a lot of synchronous updates to apply.
> It would be much more efficient, if a special bulk operation would be supported, so that all entries are sent to their respective replica nodes in one go. It would reduce the number of network roundtrips and related overhead and it would also make use of parallel execution, because some of the puts can be executed in parallel as they land on different nodes.
> Note: Hazelcast has this optimization. It often improves performance of bulk puts by at least a factor of 2 or 3, even if it is used between a client and Hazelcast server, i.e. a bulk operation is sent to a single grid node from a client and no parallelism can be used.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5442) Integration tests for Lucene Directory need an H2 database available to multiple nodes
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-5442?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero updated ISPN-5442:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Integration tests for Lucene Directory need an H2 database available to multiple nodes
> --------------------------------------------------------------------------------------
>
> Key: ISPN-5442
> URL: https://issues.jboss.org/browse/ISPN-5442
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Query
> Reporter: Sanne Grinovero
> Assignee: Sanne Grinovero
> Fix For: 8.0.0.Alpha1
>
>
> The clustered Hibernate Search tests are meant to share state using a single (external) relational database.
> The current integration tests use H2, but the tests have to be carefully written to consider the two nodes can not actually share state but only share the index.
> I'll change this to use a shared H2 instance so that people writing new tests won't have to keep these hacks in mind.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months
[JBoss JIRA] (ISPN-5442) Integration tests for Lucene Directory need an H2 database available to multiple nodes
by Sanne Grinovero (JIRA)
[ https://issues.jboss.org/browse/ISPN-5442?page=com.atlassian.jira.plugin.... ]
Sanne Grinovero reopened ISPN-5442:
-----------------------------------
> Integration tests for Lucene Directory need an H2 database available to multiple nodes
> --------------------------------------------------------------------------------------
>
> Key: ISPN-5442
> URL: https://issues.jboss.org/browse/ISPN-5442
> Project: Infinispan
> Issue Type: Task
> Components: Test Suite - Query
> Reporter: Sanne Grinovero
> Assignee: Sanne Grinovero
> Fix For: 8.0.0.Alpha1
>
>
> The clustered Hibernate Search tests are meant to share state using a single (external) relational database.
> The current integration tests use H2, but the tests have to be carefully written to consider the two nodes can not actually share state but only share the index.
> I'll change this to use a shared H2 instance so that people writing new tests won't have to keep these hacks in mind.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
9 years, 8 months