[infinispan-issues] [JBoss JIRA] (ISPN-8550) Try to estimate malloc overhead and add to memory based eviction

[Dan Berindei (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493582#comment-13493582 ] 

Dan Berindei commented on ISPN-8550:
------------------------------------

[~william.burns] how did you use Valgrind? I wanted to reproduce your results and I tried the massif tool with {{--pages-as-heap=yes}} to capture the real allocation size, but the output was kind of hard to parse. 

Eventually I found that {{$(which time) -f "%E %M"}} writes the peak RSS and I've been testing with that. After modifying the test to skip the warmup and take the size and count of allocations as parameters, this is what I got on my machine:

||alloc size||alloc count||time %M kbytes||est. entries kbytes||est. overhead 8 aligned at 16||diff||est. overhead 0 aligned at 32||diff||est.baseline kbytes||kbyte size||
|100000000|1|170336|97656.25|97656.265625|-0.015625|97656.25|0|72679.75|1024|
|1000000|1000|1053108|976562.5|976578.125|3850.125|976562.5|3865.75|
|1|1000000|104200|976.5625|15625|15895.25|31250|270.25|
|10|1000000|104160|9765.625|31250|230.25|31250|230.25|
|100|1000000|182524|97656.25|109375|469.25|125000|-15155.75|
|104|1000000|182180|101562.5|109375|125.25|125000|-15499.75|
|108|1000000|198268|105468.75|125000|588.25|125000|588.25|
|116|1000000|198060|113281.25|125000|380.25|125000|380.25|
|1000|1000000|1057348|976562.5|984375|293.25|1000000|-15331.75|
|1004|1000000|1073024|980468.75|1000000|344.25|1000000|344.25|
|1008|1000000|1073068|984375|1000000|388.25|1000000|388.25|
|1012|1000000|1073128|988281.25|1000000|448.25|1000000|448.25|
|1016|1000000|1072636|992187.5|1000000|-43.75|1000000|-43.75|
|1020|1000000|1088316|996093.75|1015625|11.25|1000000|15636.25|
|1024|1000000|1088564|1000000|1015625|259.25|1000000|15884.25|

So it looks like on my machine malloc adds 8 bytes and then rounds up the allocation to a multiple of 16, except the minimum size is 32 bytes.

This is my modified test:

{code:java}
   public static void main(String[] args) throws InterruptedException {
      int allocationSize = Integer.parseInt(args[0]);
      int allocationCount = Integer.parseInt(args[1]);
      System.out.println(allocationSize + " " + allocationCount);

      for (int i = 0; i < allocationCount; ++i) {
         long address = OffHeapMemory.INSTANCE.allocate(allocationSize);
         for (int j = 0; j < allocationSize; j += 1024) {
            OffHeapMemory.INSTANCE.putByte(address, j, (byte) 0);
         }
      }

      // Make sure Valgrind/Massif sees a the peak here
      long address = OffHeapMemory.INSTANCE.allocate(1);
      OffHeapMemory.INSTANCE.free(address);

      Thread.sleep(200);
   }
{code}

> Try to estimate malloc overhead and add to memory based eviction
> ----------------------------------------------------------------
>
>                 Key: ISPN-8550
>                 URL: https://issues.jboss.org/browse/ISPN-8550
>             Project: Infinispan
>          Issue Type: Sub-task
>            Reporter: William Burns
>            Assignee: William Burns
>             Fix For: 9.2.0.Beta2, 9.1.4.Final
>
>
> We should try to also estimate malloc overhead. We could do something like Dan mentioned at https://github.com/infinispan/infinispan/pull/5590#pullrequestreview-78054779



--
This message was sent by Atlassian JIRA
(v7.5.0#75005)