[infinispan-dev] infinispan and chunkd

[Manik Surtani]

Hi Jeff

Sorry for the late reply, it's been a busy couple of weeks.  :)

I've cc'd infinispan-dev [1] as there are others who would be interested and may want to pitch in - I recommend your joining the list as well if you haven't already done so.  Anyway, comments inline:

On 5 Nov 2009, at 21:24, Jeff Garzik wrote:

> 
> Manik,
> 
> Just a quick FYI...  "chunkd", Project Hail's single node low-level data
> storage service, just received an API and network protocol update to
> make chunkd an even more general remote key-value storage service.
> 
> 	http://hail.wiki.kernel.org/index.php/Chunkd

Nice stuff!


> Two general questions...
> 
> * Is there room for chunkd to be a storage backend for Infinispan?

Absolutely.  I can see chunkd being an excellent storage node for a CacheStore, either being colocated alongside each Infinispan node (communicating over a local socket).  

Another potentially interesting and useful integration point - albeit more complex - is to use chunkd as a DataContainer [2].  You mentioned that chunkd could use a ramdisk partition delegating the memory mgmt to the kernel, thereby buying us storage outside of the JVM heap which could be interesting.  I suppose for this to make sense chunkd's operations would have to map to DataContainer - particularly, it would need to support both FIFO and LRU ordering of entries so as to be able to access entries via an iterator.  Have you thought about this?

> 
> * Is there anyone who might be talked into creating a Java
>  interface for chunkd, now that it is _very_ map-like?

Do you actually need one?  Just a Map is of limited use.  If you want a richer API, folks probably could use Infinispan on top of chunkd (assuming a ChunkdCacheStore) - Infinispan's Cache interface extends ConcurrentMap.  Or for a *really* simple API, perhaps what you need is a REST API.  I've started a REST-* caching [3] [4] standardisation effort, haven't really had time to post much there as yet but it would likely look a lot like Infinispan's own REST API [5].

Cheers
Manik

[1] https://lists.jboss.org/mailman/listinfo/infinispan-dev
[2] http://docs.jboss.org/infinispan/4.0/apidocs/org/infinispan/container/DataContainer.html
[3] http://www.jboss.org/reststar
[4] http://groups.google.com/group/reststar-caching
[5] http://www.jboss.org/community/wiki/InfinispanRESTserver



> 
>  Writing a good Java interface requires knowledge of the available
>  standards, and I am probably the worst possible choice for doing
>  that.
> 
> Regards,
> 
> 	Jeff
> 
> 
> 

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org