[infinispan-dev] Infinispan embedded off-heap cache

[Tristan Tarrant]

Hi Ben,

HugeCollections does indeed look interesting, and we'd gladly accept a 
DataContainer implementation as you propose :)

Before you start working on it, however, I'd better expand on the 
reasons for implementing an off-heap DataContainer: we would ultimately 
like to have cache entries directly accessible as (Direct)ByteBuffers 
which means we could then use NIO2 to directly transfer that data over 
the network (JGroups, HotRod, etc) without additional copy operations.
HC uses Unsafe.allocateMemory() which does not have this facility, so we 
would still have to copy data from there to a heap-based ByteBuffer 
before being able to pass that on to any NIO2 methods.

This is all in the initial planning stages, so any comments are welcome

Tristan

On 01/15/2014 11:54 AM, cotton-ben wrote:
> Hi Yavuz, Tristan, et. al.
>
> I am extremely interested to learn if anything materialized from the
> https://issues.jboss.org/browse/ISPN-871 effort.
>
> If nothing materialized, I would like to take a stab at doing this,
> specifically by doing the following:
>
>
> 0.  Use Peter Lawrey's openHFT HugeCollections project
> (https://github.com/OpenHFT/HugeCollections) as the  off-Heap Cache
> implementation provider.
>
> 1.  Start with Peter's net.openhft.collections.HugeHashMap<K,V>
> implementation Class as a highly optimized off-Heap basis and a potential
> Cache<K,V> candidate
>
> 2.  Confirm from ISPN-dev team that the ambition to use
> org.infinispan.container.DataContainer interface as a bridge to provide
> potential  non-ISPN built  CacheImpl<K,V> candidates is sound/complete (and
> intended)
>
> 3.  Modify HugheHashMap<K,V> so that it explicitly implements
> org.infinispan.container.DataContainer interface.
>
> 4.  Confirm that modified my net.openhft.collections..HugeHashMap<K,V> can
> interoperate with the full ISPN 5.3/6.x APIs, exactly as if it were a
> default ISPN-provided org.infinispan.CacheImpl<K,V>
>
> Could any one from the ISPN-dev team comment if this ambition has merit and
> a liklihood of "working" as outlined above (effectively resuming the work
> started at https://issues.jboss.org/browse/ISPN-871)?  Is there any in-place
> ISPN documentation that advocates the use of DataContainer for taking on
> this type of effort?
>
> Thanks,
> Ben
>
>
>
> --
> View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028642.html
> Sent from the Infinispan Developer List mailing list archive at Nabble.com.
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>