[infinispan-dev] Long time no hear, or ISPN-78 continued

[Olaf Bergner]

Hello,

it's been a while ... I'm the guy who a few months ago implemented 
Infinispan's Spring support and started to work on ISPN-78 - Large 
Object Support. Due to very personal reasons and a job change that 
required my undivided attention that latter endeavour went into a 
hiatus. Sorry for that.

Provided there's still interest in the community I'd like to continue 
working on ISPN-78. But first let's see where we stand today. What 
follows is in large part a reiteration of what was already said a few 
months ago, but not all of us are blessed with an infallible memory.

At https://github.com/obergner/infinispan/tree/t_ispn78 you will find a 
very simple solution based on the concept of a StreamingHandler, to be 
obtained via Cache.getStreamingHandler():

public interface StreamingHandler<K> {
void writeToKey(K key, InputStream largeObject);
OutputStream writeToKey(K key);
InputStream readFromKey(K key);
boolean removeKey(K key);
StreamingHandler<K> withFlags(Flag... flags);
}


This solution basically works yet needs some love before it could be 
left out into the wild. I think it's not too bad but in the interim I 
have come to prefer a different solution I would like to start working 
on, namely a dedicated StreamingCache that's completely separate from 
the regular Cache.

Pros:

1. I think that it's highly unlikely that users would want store large 
objects as well as "regular" objects within the same cache instance. In 
my experience these two classes of objects are treated entirely 
differently on the application level.

2. Moreover I *suspect* - though I know next to nothing about these 
matters - that a user would have a hard time finding a set of tuning 
parameters to satisfy the needs of both regular as well as large objects.

3. Keeping large object support as part of the regular cache *might* 
entail that the same code paths would have to be optimized for both 
regular as well as large objects. This *could* prove difficult in the 
long term.

4. A separate StreamingCache would open up the possibility of defining 
default settings tailored to the needs of large objects. Furthermore I 
*suspect* that many cache settings would be irrelevant for large objects.

5. The semantics of regular and large object caches are sufficiently 
different to warrant a clean separation. Large objects will very likely 
not be replicated across different nodes, and I *suspect* that we will 
have a hard time supporting full-blown transactions for them.

Cons:

1. Increased complexity.

2. Increased maintenance burden. Well, maybe. On the one hand, there's 
one more kind of cache to take care of. On the other hand the regular 
cache wouldn't have to pay attention to the streaming cache's needs.

3. Higher implementation effort ;-)

So my proposed course of action is:

1. Introduce a new interface StreamingCache that looks exactly like the 
StreamingHandler above.

2. Rename StreamingHandlerImpl to StreamingCacheImpl.

3. Introduce a new concept, ChunkStore, a store for - you guessed it - 
chunks.

4. Modify StreamingCacheImpl to delegate to ChunkStore instead of Cache.

5. And now for the fun part: create StreamingCacheConfiguration, a 
basic, stripped down, bare bones, no frills variant of Configuration. 
Whoa, that configuration code sure looks ... interesting. Could probably 
need some pointers here.

6. Pray that I won't need a custom DataContainer and so forth.

What do you think? Does that make sense? Does it make basically sense, 
yet you would suggest some improvements? Is it utter nonsense?

The current implementation basically works and entails minimal changes 
to the existing code base. The proposed solution is not exactly rocket 
science, yet considerably more work. So I'd like to make sure that I 
don't head in the wrong direction.

Cheers,
Olaf

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20110915/b55d45f5/attachment.html