[infinispan-dev] [ISPN-78] RFC: Finalizing API

Manik Surtani manik at jboss.org
Wed May 18 06:56:02 EDT 2011


Sorry I haven't responded on this thread for so long - I'm finally here though!  :)

On 26 Apr 2011, at 21:17, Olaf Bergner wrote:

> Continuing to work on large object support I'm now at a point where I 
> would like to finalize the API so that I'm free to move forward with 
> some confidence. This is its current incarnation
> 
> public interface StreamingHandler<K> {
> 
>    void writeToKey(K key, InputStream largeObject);
> 
>    OutputStream writeToKey(K key);
> 
>    InputStream readFromKey(K key);
> 
>    boolean removeKey(K key);
> 
>    StreamingHandler<K> withFlags(Flag... flags);
> }
> 
> where a user obtains a StreamingHandler through calling 
> Cache.getStreamingHandler(). The StreamingHandler manages large objects 
> on behalf of the backing cache. This doesn't look too bad to me, but 
> there's always room for improvement.
> 
> First a fundamental question with potentially disruptive implications: 
> what aspires Large Object Support to become in the long run? A 
> comfortable means for users to store and retrieve large objects in 
> Infinspan (as it seems today)? Or rather a fully fledged distributed 
> file system? I favor the former, without precluding the possibility that 
> one day Infinispan will also implement a file system interface.

I agree with you - the initial goal would be to enable easy storage and retrieval of large objects, and also the ability to store/retrieve objects that exceed the size of a single node's JVM.  E.g., consider a grid of 100 nodes, each with 1GB heap, and the requirement to store a 4GB object (say, a DVD) in the grid which should have a theoretical capacity of 50GB (assuming 2 owners, and no overhead).

Using Infinispan as the backing tech to a file system interface is just syntactic sugar - if you want to call it that - over the use cases above.  So it should be assumed that someone will try and do this at some point in time.  :)

> Now a question regarding the implementation the answer to which might 
> affect the API: does it make sense to strictly separate "regular" caches 
> from those dealing with large objects? I think so, since I presume that 
> most applications will treat large objects differently from the more 
> comfortably sized ones. At least that is my personal experience. 
> Furthermore it might prove difficult to tune a cache that contains both 
> regular and large objects. Plus by introducing large object caches we 
> might be able to find a nice set of default settings for those.
> If we chose to introduce dedicated large object caches I would opt for 
> introducing StreamingCache<K> or even LargeObjectCache<K> instead of 
> StreamingHandler<K> since then a StreamingHandler wouldn't handle large 
> objects on behalf of some backing cache. Rather it would act as *the* 
> interface to a cache exclusively reserved for large objects. It follows 
> that a user would directly access a StreamingCache, not indirectly via 
> Cache.getStreamingHandler().

Good point.  I too don't foresee people using the same cache instance for regular objects as well as large objects.  What do you propose this StreamingCache interface look like?  And how would one get a hold of it?  Is it just a decorator, constructed from a regular cache, such as:

StreamingCache sc = new StreamingCache(myRegularCache);

> And finally there is that eternal question of how to properly name those 
> interface methods. Trustin has expressed his concern that 
> writeToKey(...), removeKey() and readFromKey(...) might fail to convey 
> their semantics. And indeed there's nothing in their names to tell a 
> user that they deal with large objects. What about alternatives á là
> 
> - void storeLargeObject(K key, InputStream largeObject) or 
> putLargeObject(K key, InputStream largeObject)
> 
> - OutputStream newLargeObject(K key) or simply OutputStream 
> largeObject(K key)
> 
> - InputStream getLargeObject(K key)
> 
> - boolean removeLargeObject(K key)

Suggestions above are good; here are a few more choices:

void storeFromStream(K key, InputStream is);
OutputStream openStreamForWriting(K key);
InputStream openStreamForReading(K key);
boolean remove(K key);

Cheers
Manik
--
Manik Surtani
manik at jboss.org
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org






More information about the infinispan-dev mailing list