[
https://issues.jboss.org/browse/ISPN-8550?page=com.atlassian.jira.plugin....
]
Dan Berindei edited comment on ISPN-8550 at 11/28/17 4:48 AM:
--------------------------------------------------------------
Adding the links we discussed last night for future reference:
* [glibc/ptmalloc2 sources|http://repo.or.cz/glibc.git/blob/HEAD:/malloc/malloc.c#l91] say
it indeed aligns to 16 bytes (4096 bytes for allocations >= 128KB) after adding 8 bytes
for the block size, with a minimum allocation size of 32. May be prone to fragmentation
after lots of allocations with different sizes.
* [
jemalloc|http://jemalloc.net/jemalloc.3.html] doesn't need to store the block size,
but has an alignment overhead of up to 20% (up to 50% for allocations < 80 bytes). It
may be better at handling fragmentation in the long run.
* [
tcmalloc|https://github.com/gperftools/gperftools/blob/master/src/common....] has
more size classes than jemalloc, and an alignment overhead of up to 12.5% (or 7 bytes for
allocations < 128 bytes). It is supposed to be much faster than ptmalloc2, but there
are reports that it doesn't handle fragmentation well (it is used by Chrome).
All allocators have per-thread caches, which makes things faster but may increase
fragmentation (see
https://github.com/gperftools/gperftools/issues/756). They also have
lots of compile options, so we may want to experiment with bundling a custom-built
implementation -- assuming we don't switch to mmap-ing an arena and doing the space
management in Java-land first.
was (Author: dan.berindei):
Adding the links we discussed last night for future reference:
* [glibc/ptmalloc2 sources|http://repo.or.cz/glibc.git/blob/HEAD:/malloc/malloc.c#l91] say
it indeed aligns to 16 bytes (4096 bytes for allocations >= 128KB) after adding 8 bytes
for the block size, with a minimum allocation size of 32. May be prone to fragmentation
after lots of allocations with different sizes.
* [
jemalloc|http://jemalloc.net/jemalloc.3.html] doesn't need to store the block size,
but has an alignment overhead of up to 20% (more only for allocations < 80 bytes). It
may be better at handling fragmentation in the long run.
* [
tcmalloc|https://github.com/gperftools/gperftools/blob/master/src/common....] has
more size classes than jemalloc, and an alignment overhead of up to 12.5% (or 7 bytes for
allocations < 128 bytes). It is supposed to be much faster than ptmalloc2, but there
are reports that it doesn't handle fragmentation well (it is used by Chrome).
All allocators have per-thread caches, which makes things faster but may increase
fragmentation (see
https://github.com/gperftools/gperftools/issues/756). They also have
lots of compile options, so we may want to experiment with bundling a custom-built
implementation -- assuming we don't switch to mmap-ing an arena and doing the space
management in Java-land first.
Try to estimate malloc overhead and add to memory based eviction
----------------------------------------------------------------
Key: ISPN-8550
URL:
https://issues.jboss.org/browse/ISPN-8550
Project: Infinispan
Issue Type: Sub-task
Reporter: William Burns
Assignee: William Burns
Fix For: 9.2.0.Beta2, 9.1.4.Final
Attachments: allocs.ods
We should try to also estimate malloc overhead. We could do something like Dan mentioned
at
https://github.com/infinispan/infinispan/pull/5590#pullrequestreview-7805...
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)