[infinispan-dev] Long time no hear, or ISPN-78 continued

[Olaf Bergner]

Am 04.10.11 20:15, schrieb Mircea Markus:
> On 15 Sep 2011, at 17:50, Olaf Bergner wrote:
>
>> Hello,
>>
>> it's been a while ... I'm the guy who a few months ago implemented 
>> Infinispan's Spring support and started to work on ISPN-78 - Large 
>> Object Support. Due to very personal reasons and a job change that 
>> required my undivided attention that latter endeavour went into a 
>> hiatus. Sorry for that.
>>
>> Provided there's still interest in the community I'd like to continue 
>> working on ISPN-78. But first let's see where we stand today. What 
>> follows is in large part a reiteration of what was already said a few 
>> months ago, but not all of us are blessed with an infallible memory.
>>
>> At https://github.com/obergner/infinispan/tree/t_ispn78 you will find 
>> a very simple solution based on the concept of a StreamingHandler, to 
>> be obtained via Cache.getStreamingHandler():
>>
>> public interface StreamingHandler<K> {
>> void writeToKey(K key, InputStream largeObject);
>> OutputStream writeToKey(K key);
>> InputStream readFromKey(K key);
>> boolean removeKey(K key);
>> StreamingHandler<K> ; withFlags(Flag... flags );
>> }
>>
>> This solution basically works yet needs some love before it could be 
>> left out into the wild. I think it's not too bad but in the interim I 
>> have come to prefer a different solution I would like to start 
>> working on, namely a dedicated StreamingCache that's completely 
>> separate from the regular Cache.
> I guess you'd  obtain a StreamingCache from the CacheManager rather 
> than obtaining it from a Cache?
Yes, that's my current line of thinking, at least.
>>
>> Pros:
>>
>> 1. I think that it's highly unlikely that users would want store 
>> large objects as well as "regular" objects within the same cache 
>> instance. In my experience these two classes of objects are treated 
>> entirely differently on the application level.
>>
>> 2. Moreover I *suspect* - though I know next to nothing about these 
>> matters - that a user would have a hard time finding a set of tuning 
>> parameters to satisfy the needs of both regular as well as large objects.
>>
>> 3. Keeping large object support as part of the regular cache *might* 
>> entail that the same code paths would have to be optimized for both 
>> regular as well as large objects. This *could* prove difficult in the 
>> long term.
>>
>> 4. A separate StreamingCache would open up the possibility of 
>> defining default settings tailored to the needs of large objects. 
>> Furthermore I *suspect* that many cache settings would be irrelevant 
>> for large objects.
>>
>> 5. The semantics of regular and large object caches are sufficiently 
>> different to warrant a clean separation. Large objects will very 
>> likely not be replicated across different nodes, and I *suspect* that 
>> we will have a hard time supporting full-blown transactions for them.
>>
>> Cons:
>>
>> 1. Increased complexity.
>>
>> 2. Increased maintenance burden. Well, maybe. On the one hand, 
>> there's one more kind of cache to take care of. On the other hand the 
>> regular cache wouldn't have to pay attention to the streaming cache's 
>> needs.
>>
>> 3. Higher implementation effort ;-)
>>
>> So my proposed course of action is:
>>
>> 1. Introduce a new interface StreamingCache that looks exactly like 
>> the StreamingHandler above.
>>
>> 2. Rename StreamingHandlerImpl to StreamingCacheImpl.
>>
>> 3. Introduce a new concept, ChunkStore, a store for - you guessed it 
>> - chunks.
>>
>> 4. Modify StreamingCacheImpl to delegate to ChunkStore instead of Cache.
> Why not create a fully flagged Cache under the hood and use that from 
> the StreamingCacheImpl - this way you'd be able to reuse the current 
> implementation. Same for the configuration, the StreamingCacheConfig 
> would basically wrap an Configuration + some additional fields.
It sounds like something I should at least try as it significantly 
simplifies things. Maybe the devil's in the detail, but I will probably 
start down that route.
>>
>> 5. And now for the fun part: create StreamingCacheConfiguration, a 
>> basic, stripped down, bare bones, no frills variant of Configuration. 
>> Whoa, that configuration code sure looks ... interesting. Could 
>> probably need some pointers here.
>>
>> 6. Pray that I won't need a custom DataContainer and so forth.
> you might actually, as I imagine one might want to use eviction and 
> passivation etc.. That would be solved with the cache approach I 
> described previously.
Well, that cache approach just grew even more compelling.
>>
>> What do you think? Does that make sense? Does it make basically 
>> sense, yet you would suggest some improvements? Is it utter nonsense?
> +1 i.e. makes total sense to me.
OK, then I'll give this a try. Thx.
>>
>> The current implementation basically works and entails minimal 
>> changes to the existing code base. The proposed solution is not 
>> exactly rocket science, yet considerably more work. So I'd like to 
>> make sure that I don't head in the wrong direction.
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20111004/9a569dba/attachment-0001.html