[infinispan-issues] [JBoss JIRA] (ISPN-3877) SingleFileStore continues to consume space on the disk even if the free entries at end of the file are not in use. This space can be released back to the filesystem

[Rajesh Jangam (JIRA)]

     [ https://issues.jboss.org/browse/ISPN-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajesh Jangam updated ISPN-3877:
--------------------------------

        Summary: SingleFileStore continues to consume space on the disk even if the free entries at end of the file are not in use. This space can be released back to the filesystem  (was: Disk Space optimizations for SingleFileStore)
    Description: 
We are testing the latest Infinispan 6.0 SingleFileStore version for our application.
We tend to create a lot of entries in the store and remove them when not required.
However, what we have observed is that the size of the file keeps on increasing as more data is being added/removed and that it does not get reclaimed.
 
This was not seen with earlier versions like 5.3.0. There the size of the cache folder would reduce as entries got removed.

We have attempted to create a fix for this issue and tested it in our environment. It keeps the space usage under control as compared to the original implementation.

As a part of this fix, we truncate the file (as a part of purge cycle) if there are free entries found towards the end of the file. Also these free entries are removed from the freeList.



  was:
We are testing the latest Infinispan 6.0 SingleFileStore version for our application.
We tend to create a lot of entries in the store and remove them when not required.
However, what we have observed is that the size of the file keeps on increasing as more data is being added/removed and that it does not get reclaimed.
 
This was not seen with earlier versions like 5.3.0. There the size of the cache folder would reduce as entries got removed.


We have attempted to create a fix for this issue and tested it in our environment. It keeps the space usage under control as compared to the original implementation.

The proposed fix is twofold:

1. Truncates the file if there are free fileentries at the end of the data file.
2. Coalesces consecutive free fileentries into one so that the probability of finding a free file entry capable enough to contain a newly added key-value pair increases.

Please review this diff and let us know your opinion


    
> SingleFileStore continues to consume space on the disk even if the free entries at end of the file are not in use. This space can be released back to the filesystem
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: ISPN-3877
>                 URL: https://issues.jboss.org/browse/ISPN-3877
>             Project: Infinispan
>          Issue Type: Patch
>          Components: Loaders and Stores
>    Affects Versions: 6.0.0.Final
>         Environment: Oracle JDK 1.7, Windows Server 2008 R2 x64
>            Reporter: Rajesh Jangam
>            Assignee: Galder Zamarreño
>
> We are testing the latest Infinispan 6.0 SingleFileStore version for our application.
> We tend to create a lot of entries in the store and remove them when not required.
> However, what we have observed is that the size of the file keeps on increasing as more data is being added/removed and that it does not get reclaimed.
>  
> This was not seen with earlier versions like 5.3.0. There the size of the cache folder would reduce as entries got removed.
> We have attempted to create a fix for this issue and tested it in our environment. It keeps the space usage under control as compared to the original implementation.
> As a part of this fix, we truncate the file (as a part of purge cycle) if there are free entries found towards the end of the file. Also these free entries are removed from the freeList.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira