[infinispan-dev] Infinispan embedded off-heap cache

[Tristan Tarrant]

The Netty off-heap allocator (or rather a Java port of jemalloc) is 
exactly the sort of thing I had in mind.

Tristan

On 01/15/2014 12:59 PM, Jaromir Hamala wrote:
> Hi,
>
> another option is to use the off-heap allocator from Netty project. 
> It's just an allocator, so it would require extra work when compared 
> with HugeCollections.
> I tried this approach approach with Hazelcast: 
> https://github.com/jerrinot/hugecast
>
> Cheers,
> Jaromir
>
>
> On Wed, Jan 15, 2014 at 11:44 AM, Tristan Tarrant <ttarrant at redhat.com 
> <mailto:ttarrant at redhat.com>> wrote:
>
>     Hi Ben,
>
>     HugeCollections does indeed look interesting, and we'd gladly accept a
>     DataContainer implementation as you propose :)
>
>     Before you start working on it, however, I'd better expand on the
>     reasons for implementing an off-heap DataContainer: we would
>     ultimately
>     like to have cache entries directly accessible as (Direct)ByteBuffers
>     which means we could then use NIO2 to directly transfer that data over
>     the network (JGroups, HotRod, etc) without additional copy operations.
>     HC uses Unsafe.allocateMemory() which does not have this facility,
>     so we
>     would still have to copy data from there to a heap-based ByteBuffer
>     before being able to pass that on to any NIO2 methods.
>
>     This is all in the initial planning stages, so any comments are
>     welcome
>
>     Tristan
>
>     On 01/15/2014 11:54 AM, cotton-ben wrote:
>     > Hi Yavuz, Tristan, et. al.
>     >
>     > I am extremely interested to learn if anything materialized from the
>     > https://issues.jboss.org/browse/ISPN-871 effort.
>     >
>     > If nothing materialized, I would like to take a stab at doing this,
>     > specifically by doing the following:
>     >
>     >
>     > 0. Use Peter Lawrey's openHFT HugeCollections project
>     > (https://github.com/OpenHFT/HugeCollections) as the off-Heap Cache
>     > implementation provider.
>     >
>     > 1. Start with Peter's net.openhft.collections.HugeHashMap<K,V>
>     > implementation Class as a highly optimized off-Heap basis and a
>     potential
>     > Cache<K,V> candidate
>     >
>     > 2. Confirm from ISPN-dev team that the ambition to use
>     > org.infinispan.container.DataContainer interface as a bridge to
>     provide
>     > potential non-ISPN built CacheImpl<K,V> candidates is
>     sound/complete (and
>     > intended)
>     >
>     > 3. Modify HugheHashMap<K,V> so that it explicitly implements
>     > org.infinispan.container.DataContainer interface.
>     >
>     > 4. Confirm that modified my
>     net.openhft.collections..HugeHashMap<K,V> can
>     > interoperate with the full ISPN 5.3/6.x APIs, exactly as if it
>     were a
>     > default ISPN-provided org.infinispan.CacheImpl<K,V>
>     >
>     > Could any one from the ISPN-dev team comment if this ambition
>     has merit and
>     > a liklihood of "working" as outlined above (effectively resuming
>     the work
>     > started at https://issues.jboss.org/browse/ISPN-871)? Is there
>     any in-place
>     > ISPN documentation that advocates the use of DataContainer for
>     taking on
>     > this type of effort?
>     >
>     > Thanks,
>     > Ben
>     >
>     >
>     >
>     > --
>     > View this message in context:
>     http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028642.html
>     > Sent from the Infinispan Developer List mailing list archive at
>     Nabble.com.
>     > _______________________________________________
>     > infinispan-dev mailing list
>     > infinispan-dev at lists.jboss.org
>     <mailto:infinispan-dev at lists.jboss.org>
>     > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>     >
>     >
>
>     _______________________________________________
>     infinispan-dev mailing list
>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
>
>
> -- 
> “Perfection is achieved, not when there is nothing more to add, but 
> when there is nothing left to take away.”
> Antoine de Saint Exupéry
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev