Re: [infinispan-dev] Long time no hear, or ISPN-78 continued

On 15 Sep 2011, at 17:50, Olaf Bergner wrote: > Hello, > > it's been a while ... I'm the guy who a few months ago implemented > Infinispan's Spring support and started to work on ISPN-78 - Large > Object Support. Due to very personal reasons and a job change that > required my undivided attention that latter endeavour went into a > hiatus. Sorry for that. > > Provided there's still interest in the community I'd like to continue > working on ISPN-78. But first let's see where we stand today. What > follows is in large part a reiteration of what was already said a few > months ago, but not all of us are blessed with an infallible memory. > > At https://github.com/obergner/infinispan/tree/t_ispn78 you will find > a very simple solution based on the concept of a StreamingHandler, to > be obtained via Cache.getStreamingHandler(): > > public interface StreamingHandler<K> { > void writeToKey(K key, InputStream largeObject); > OutputStream writeToKey(K key); > InputStream readFromKey(K key); > boolean removeKey(K key); > StreamingHandler<K> ; withFlags(Flag... flags ); > } > > This solution basically works yet needs some love before it could be > left out into the wild. I think it's not too bad but in the interim I > have come to prefer a different solution I would like to start > working on, namely a dedicated StreamingCache that's completely > separate from the regular Cache. I guess you'd obtain a StreamingCache from the CacheManager rather than obtaining it from a Cache?

> > Pros: > > 1. I think that it's highly unlikely that users would want store > large objects as well as "regular" objects within the same cache > instance. In my experience these two classes of objects are treated > entirely differently on the application level. > > 2. Moreover I *suspect* - though I know next to nothing about these > matters - that a user would have a hard time finding a set of tuning > parameters to satisfy the needs of both regular as well as large objects. > > 3. Keeping large object support as part of the regular cache *might* > entail that the same code paths would have to be optimized for both > regular as well as large objects. This *could* prove difficult in the > long term. > > 4. A separate StreamingCache would open up the possibility of > defining default settings tailored to the needs of large objects. > Furthermore I *suspect* that many cache settings would be irrelevant > for large objects. > > 5. The semantics of regular and large object caches are sufficiently > different to warrant a clean separation. Large objects will very > likely not be replicated across different nodes, and I *suspect* that > we will have a hard time supporting full-blown transactions for them. > > Cons: > > 1. Increased complexity. > > 2. Increased maintenance burden. Well, maybe. On the one hand, > there's one more kind of cache to take care of. On the other hand the > regular cache wouldn't have to pay attention to the streaming cache's > needs. > > 3. Higher implementation effort ;-) > > So my proposed course of action is: > > 1. Introduce a new interface StreamingCache that looks exactly like > the StreamingHandler above. > > 2. Rename StreamingHandlerImpl to StreamingCacheImpl. > > 3. Introduce a new concept, ChunkStore, a store for - you guessed it > - chunks. > > 4. Modify StreamingCacheImpl to delegate to ChunkStore instead of Cache. Why not create a fully flagged Cache under the hood and use that from the StreamingCacheImpl - this way you'd be able to reuse the current implementation. Same for the configuration, the StreamingCacheConfig would basically wrap an Configuration + some additional fields.

> > 5. And now for the fun part: create StreamingCacheConfiguration, a > basic, stripped down, bare bones, no frills variant of Configuration. > Whoa, that configuration code sure looks ... interesting. Could > probably need some pointers here. > > 6. Pray that I won't need a custom DataContainer and so forth. you might actually, as I imagine one might want to use eviction and passivation etc.. That would be solved with the cache approach I described previously.

> > What do you think? Does that make sense? Does it make basically > sense, yet you would suggest some improvements? Is it utter nonsense? +1 i.e. makes total sense to me.

> > The current implementation basically works and entails minimal > changes to the existing code base. The proposed solution is not > exactly rocket science, yet considerably more work. So I'd like to > make sure that I don't head in the wrong direction. _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev