Re: [infinispan-dev] Infinispan embedded off-heap cache

Hi, another option is to use the off-heap allocator from Netty project. It's just an allocator, so it would require extra work when compared with HugeCollections. I tried this approach approach with Hazelcast: https://github.com/jerrinot/hugecast Cheers, Jaromir On Wed, Jan 15, 2014 at 11:44 AM, Tristan Tarrant <ttarrant(a)redhat.com <mailto:ttarrant@redhat.com>> wrote: Hi Ben, HugeCollections does indeed look interesting, and we'd gladly accept a DataContainer implementation as you propose :) Before you start working on it, however, I'd better expand on the reasons for implementing an off-heap DataContainer: we would ultimately like to have cache entries directly accessible as (Direct)ByteBuffers which means we could then use NIO2 to directly transfer that data over the network (JGroups, HotRod, etc) without additional copy operations. HC uses Unsafe.allocateMemory() which does not have this facility, so we would still have to copy data from there to a heap-based ByteBuffer before being able to pass that on to any NIO2 methods. This is all in the initial planning stages, so any comments are welcome Tristan On 01/15/2014 11:54 AM, cotton-ben wrote: > Hi Yavuz, Tristan, et. al. > > I am extremely interested to learn if anything materialized from the > https://issues.jboss.org/browse/ISPN-871 effort. > > If nothing materialized, I would like to take a stab at doing this, > specifically by doing the following: > > > 0. Use Peter Lawrey's openHFT HugeCollections project > (https://github.com/OpenHFT/HugeCollections) as the off-Heap Cache > implementation provider. > > 1. Start with Peter's net.openhft.collections.HugeHashMap<K,V> > implementation Class as a highly optimized off-Heap basis and a potential > Cache<K,V> candidate > > 2. Confirm from ISPN-dev team that the ambition to use > org.infinispan.container.DataContainer interface as a bridge to provide > potential non-ISPN built CacheImpl<K,V> candidates is sound/complete (and > intended) > > 3. Modify HugheHashMap<K,V> so that it explicitly implements > org.infinispan.container.DataContainer interface. > > 4. Confirm that modified my net.openhft.collections..HugeHashMap<K,V> can > interoperate with the full ISPN 5.3/6.x APIs, exactly as if it were a > default ISPN-provided org.infinispan.CacheImpl<K,V> > > Could any one from the ISPN-dev team comment if this ambition has merit and > a liklihood of "working" as outlined above (effectively resuming the work > started at https://issues.jboss.org/browse/ISPN-871)? Is there any in-place > ISPN documentation that advocates the use of DataContainer for taking on > this type of effort? > > Thanks, > Ben > > > > -- > View this message in context: http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infi... > Sent from the Infinispan Developer List mailing list archive at Nabble.com. > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/infinispan-dev -- “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” Antoine de Saint Exupéry _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev