[JBoss JIRA] (ISPN-4441) Improvements on getCacheEntry() synchronization
by Galder Zamarreño (JIRA)
Galder Zamarreño created ISPN-4441:
--------------------------------------
Summary: Improvements on getCacheEntry() synchronization
Key: ISPN-4441
URL: https://issues.jboss.org/browse/ISPN-4441
Project: Infinispan
Issue Type: Enhancement
Security Level: Public (Everyone can see)
Components: Core
Reporter: Galder Zamarreño
Assignee: Galder Zamarreño
Fix For: 7.0.0.Alpha5
- Separate GetKeyValueCommand into two, GetKeyValueCommand and GetCacheEntryCommand to have good separation and avoid conditional logic in GetKeyValueCommand.
- As indicated by Pedro in https://github.com/infinispan/infinispan/pull/2671:
{quote}
I had something simpler in mind. you could invoke create(cacheEntry) that returns a mutable copy. I think it is ok the copy to be mutable, but it you really want it immutable, you can do immutableInternalCacheEntry(create(cacheEntry))
this way you would avoid all the changes in CoreImmutables
{quote}
- Finally, sort out TODO in documentation of InternalEntryFactory
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
10 years
[JBoss JIRA] (ISPN-4440) Remove setMaxCollectorSize API from MapReduceTask
by Vladimir Blagojevic (JIRA)
Vladimir Blagojevic created ISPN-4440:
-----------------------------------------
Summary: Remove setMaxCollectorSize API from MapReduceTask
Key: ISPN-4440
URL: https://issues.jboss.org/browse/ISPN-4440
Project: Infinispan
Issue Type: Task
Security Level: Public (Everyone can see)
Components: Distributed Execution and Map/Reduce
Affects Versions: 7.0.0.Alpha4
Reporter: Vladimir Blagojevic
Assignee: Vladimir Blagojevic
Priority: Minor
During the refinement of parallel execution of M/R algorithm we introduced an abstraction maxCollectorSize on the level of MapReduceTask. The ideas was that during execution of map/combine phase, number of intermediate keys/values collected in a Collector could potentially become very large. By limiting size of collector, intermediate key/values are transferred to intermediate cache in batches before reduce phase is executed and OutOfMemoryError issues are avoided as well.
However, during the extensive performance phase Alan Field, Dan Berindei and I have concluded that maxCollectorSize set to 10000 entries gives the best trade off between performance and memory use. Therefore there is no need to expose this value to MapReduceTask users.
Having said that there might be some uses cases where holding 10000 intermediate large memory footprint objects might lead to OOM, and in such cases users should allocate more heap to MapReduceTasks. We might consider introducing again this API should such a need arise.
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
10 years
[JBoss JIRA] (ISPN-4439) enhance asynchronous Hot Rod putAllAsync method
by Wolf-Dieter Fink (JIRA)
[ https://issues.jboss.org/browse/ISPN-4439?page=com.atlassian.jira.plugin.... ]
Wolf-Dieter Fink updated ISPN-4439:
-----------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2672
There are some concerns
- client might be overloaded by having too many parallel threads
- server might be overloaded as all put operations are parallel
- network traffic as each key/value is send with a single remote invocation
the performance might be better if the client analyze which
server will be the owner and send the entries in chunk's of 10|100?
Also it might be worth to have a list of entries which are not processed.
> enhance asynchronous Hot Rod putAllAsync method
> ------------------------------------------------
>
> Key: ISPN-4439
> URL: https://issues.jboss.org/browse/ISPN-4439
> Project: Infinispan
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Affects Versions: 7.0.0.Alpha4
> Reporter: Wolf-Dieter Fink
> Assignee: Galder Zamarreño
>
> The current implementation of the method putAllAsync(...) within the Hot Rod client simple use the putAll(...) method which simple iterate over the given Map and use put(...) for each entry synchrounous.
> Therefore it takes a longer time to put all entries into the cache.
> A simple loop which calls putAsync for each entry has a significant improvement.
> i.e. with 100,000 entries the over all duration is 15 times faster
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
10 years
[JBoss JIRA] (ISPN-4439) enhance asynchronous Hot Rod putAllAsync method
by Wolf-Dieter Fink (JIRA)
[ https://issues.jboss.org/browse/ISPN-4439?page=com.atlassian.jira.plugin.... ]
Wolf-Dieter Fink updated ISPN-4439:
-----------------------------------
Priority: Minor (was: Major)
> enhance asynchronous Hot Rod putAllAsync method
> ------------------------------------------------
>
> Key: ISPN-4439
> URL: https://issues.jboss.org/browse/ISPN-4439
> Project: Infinispan
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Affects Versions: 7.0.0.Alpha4
> Reporter: Wolf-Dieter Fink
> Assignee: Galder Zamarreño
> Priority: Minor
>
> The current implementation of the method putAllAsync(...) within the Hot Rod client simple use the putAll(...) method which simple iterate over the given Map and use put(...) for each entry synchrounous.
> Therefore it takes a longer time to put all entries into the cache.
> A simple loop which calls putAsync for each entry has a significant improvement.
> i.e. with 100,000 entries the over all duration is 15 times faster
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
10 years
[JBoss JIRA] (ISPN-4439) enhance asynchronous Hot Rod putAllAsync method
by Wolf-Dieter Fink (JIRA)
[ https://issues.jboss.org/browse/ISPN-4439?page=com.atlassian.jira.plugin.... ]
Wolf-Dieter Fink updated ISPN-4439:
-----------------------------------
Assignee: Galder Zamarreño (was: Mircea Markus)
> enhance asynchronous Hot Rod putAllAsync method
> ------------------------------------------------
>
> Key: ISPN-4439
> URL: https://issues.jboss.org/browse/ISPN-4439
> Project: Infinispan
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Affects Versions: 7.0.0.Alpha4
> Reporter: Wolf-Dieter Fink
> Assignee: Galder Zamarreño
>
> The current implementation of the method putAllAsync(...) within the Hot Rod client simple use the putAll(...) method which simple iterate over the given Map and use put(...) for each entry synchrounous.
> Therefore it takes a longer time to put all entries into the cache.
> A simple loop which calls putAsync for each entry has a significant improvement.
> i.e. with 100,000 entries the over all duration is 15 times faster
--
This message was sent by Atlassian JIRA
(v6.2.6#6264)
10 years