[infinispan-issues] [JBoss JIRA] (ISPN-12011) Investigate CompactionFilter for purging expired entries in RocksDB

[Dan Berindei (Jira)]

     [ https://issues.redhat.com/browse/ISPN-12011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dan Berindei updated ISPN-12011:
--------------------------------
    Description: 
RocksDB allows the application to process/delete entries during compaction, and we could use this [CompactionFilter|https://github.com/facebook/rocksdb/blob/master/include/rocksdb/compaction_filter.h] callback to implement the purging of expired entries.

It's not straightforward, through: {{CompactionFilter}} must be implemented in a native (C++) library. {{rocksdb-jni}} allows using a native implementation from Java, but it does not allow defining it in Java (like it does for comparators).

Potential solutions:
 # Write a native library implementing {{CompactionFilter}} for our current storage format
 # Change the value format to match the Cassandra storage format and use the predefined {{CassandraCompactionFilter}}
 # Revive the now-removed [LuaCompactionFilter|https://github.com/facebook/rocksdb/wiki/Lua-CompactionFilter] and write our filter implementation in Lua
 # Contribute an adapter for Java CompactionFilter implementations to the RocksDB project, inspired by the existing AbstractComparator

  was:
RocksDB allows the application to process/delete entries during compaction, and we could use this {{}}[CompactionFilter|https://github.com/facebook/rocksdb/blob/master/include/rocksdb/compaction_filter.h] callback to implement the purging of expired entries.

It's not straightforward, through: {{CompactionFilter}} must be implemented in a native (C++) library. {{rocksdb-jni}} allows using a native implementation from Java, but it does not allow defining it in Java (like it does for comparators).

Potential solutions:
 # Write a native library implementing {{CompactionFilter}} for our current storage format
 # Change the value format to match the Cassandra storage format and use the predefined {{CassandraCompactionFilter}} 
 # Revive the now-removed [LuaCompactionFilter|https://github.com/facebook/rocksdb/wiki/Lua-CompactionFilter] and write our filter implementation in Lua
 # Contribute an adapter for Java CompactionFilter implementations to the RocksDB project, inspired by the existing AbstractComparator



> Investigate CompactionFilter for purging expired entries in RocksDB
> -------------------------------------------------------------------
>
>                 Key: ISPN-12011
>                 URL: https://issues.redhat.com/browse/ISPN-12011
>             Project: Infinispan
>          Issue Type: Sub-task
>          Components: Loaders and Stores
>            Reporter: Dan Berindei
>            Priority: Major
>             Fix For: 12.0.0.Final
>
>
> RocksDB allows the application to process/delete entries during compaction, and we could use this [CompactionFilter|https://github.com/facebook/rocksdb/blob/master/include/rocksdb/compaction_filter.h] callback to implement the purging of expired entries.
> It's not straightforward, through: {{CompactionFilter}} must be implemented in a native (C++) library. {{rocksdb-jni}} allows using a native implementation from Java, but it does not allow defining it in Java (like it does for comparators).
> Potential solutions:
>  # Write a native library implementing {{CompactionFilter}} for our current storage format
>  # Change the value format to match the Cassandra storage format and use the predefined {{CassandraCompactionFilter}}
>  # Revive the now-removed [LuaCompactionFilter|https://github.com/facebook/rocksdb/wiki/Lua-CompactionFilter] and write our filter implementation in Lua
>  # Contribute an adapter for Java CompactionFilter implementations to the RocksDB project, inspired by the existing AbstractComparator



--
This message was sent by Atlassian Jira
(v7.13.8#713008)