[infinispan-dev] ISPN-78 and Large Object Support/Streaming API

[Manik Surtani]

On 25 May 2010, at 12:02, Bela Ban wrote:

> In general, we need to define how LOS differs from GridFS, and when 
> which tech should be used. I see the following 3 use cases:
> 
>   1. Many smaller objects: Infinispan
>   2. Many smaller object and few very large objects: LOS in Infinispan
>   3. Many large objects and / or file system API: GridFS
> 
> OTOH, #2 could be done by creating a GridFS instance over the existing 
> Infinispan cache and using it for very large objects.
> 
> One could argue that instead of polluting the Infinispan API by adding 
> more methods, one could resort to using GridFS, in other words: GridFS 
> == LOS.
> 
> Let's try to avoid duplication of functionality here

I see it as such:

* Many small objects: standard Infinispan
* Some LOs: Infinispan LO mode (which allows access to streams + chunking)
* Explicit file-system-like behaviour: GridFS

And I see GridFS being implemented on top of Infinispan LO mode - i.e., directly interfacing with the stream API.  The difference here is that GridFS becomes syntactic sugar and a java.io.File-like handle.

It's not duplication, per-se.  More like moving a few bits of code around.  For example, I see merit in moving the chunking code from the GridFile class to an Interceptor so it can be more generically useful to arbitrary streams to the cache.

Cheers
Manik

> 
> 
> Manik Surtani wrote:
>> I have put together a brief design for ISPN-78. Please take a look, it 
>> is on the wiki:
>> 
>> https://community.jboss.org/wiki/LargeObjectSupport
>> 
>> I have also deferred ISPN-78 to 5.0.0 rather than 4.1.0 as I'd rather 
>> not hold up 4.1.0 for new features at this stage.
>> 
>> Please have a look at the designs and let me know what you think - or 
>> comment on the wiki page.
> 
> -- 
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org