Hi Olaf. I think there is still interest. I'm a bit swamped with stuff at the moment, I'll try and respond in more detail later this week, but in the meanwhile, does anyone else have any opinions? Bela? Ales? Cheers Manik On 15 Sep 2011, at 09:50, Olaf Bergner wrote: > Hello, > > it's been a while ... I'm the guy who a few months ago implemented Infinispan's Spring support and started to work on ISPN-78 - Large Object Support. Due to very personal reasons and a job change that required my undivided attention that latter endeavour went into a hiatus. Sorry for that. > > Provided there's still interest in the community I'd like to continue working on ISPN-78. But first let's see where we stand today. What follows is in large part a reiteration of what was already said a few months ago, but not all of us are blessed with an infallible memory. > > At https://github.com/obergner/infinispan/tree/t_ispn78 you will find a very simple solution based on the concept of a StreamingHandler, to be obtained via Cache.getStreamingHandler(): > > public interface StreamingHandler<K> { > > void writeToKey(K key, InputStream largeObject); > > OutputStream writeToKey(K key); > > InputStream readFromKey(K key); > > boolean removeKey(K key); > > StreamingHandler<K> > ; withFlags(Flag... flags > ); > } > > This solution basically works yet needs some love before it could be left out into the wild. I think it's not too bad but in the interim I have come to prefer a different solution I would like to start working on, namely a dedicated StreamingCache that's completely separate from the regular Cache. > > Pros: > > 1. I think that it's highly unlikely that users would want store large objects as well as "regular" objects within the same cache instance. In my experience these two classes of objects are treated entirely differently on the application level. > > 2. Moreover I *suspect* - though I know next to nothing about these matters - that a user would have a hard time finding a set of tuning parameters to satisfy the needs of both regular as well as large objects. > > 3. Keeping large object support as part of the regular cache *might* entail that the same code paths would have to be optimized for both regular as well as large objects. This *could* prove difficult in the long term. > > 4. A separate StreamingCache would open up the possibility of defining default settings tailored to the needs of large objects. Furthermore I *suspect* that many cache settings would be irrelevant for large objects. > > 5. The semantics of regular and large object caches are sufficiently different to warrant a clean separation. Large objects will very likely not be replicated across different nodes, and I *suspect* that we will have a hard time supporting full-blown transactions for them. > > Cons: > > 1. Increased complexity. > > 2. Increased maintenance burden. Well, maybe. On the one hand, there's one more kind of cache to take care of. On the other hand the regular cache wouldn't have to pay attention to the streaming cache's needs. > > 3. Higher implementation effort ;-) > > So my proposed course of action is: > > 1. Introduce a new interface StreamingCache that looks exactly like the StreamingHandler above. > > 2. Rename StreamingHandlerImpl to StreamingCacheImpl. > > 3. Introduce a new concept, ChunkStore, a store for - you guessed it - chunks. > > 4. Modify StreamingCacheImpl to delegate to ChunkStore instead of Cache. > > 5. And now for the fun part: create StreamingCacheConfiguration, a basic, stripped down, bare bones, no frills variant of Configuration. Whoa, that configuration code sure looks ... interesting. Could probably need some pointers here. > > 6. Pray that I won't need a custom DataContainer and so forth. > > What do you think? Does that make sense? Does it make basically sense, yet you would suggest some improvements? Is it utter nonsense? > > The current implementation basically works and entails minimal changes to the existing code base. The proposed solution is not exactly rocket science, yet considerably more work. So I'd like to make sure that I don't head in the wrong direction. > > Cheers, > Olaf > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 15 Sep 2011, at 17:50, Olaf Bergner wrote: > Hello, > > it's been a while ... I'm the guy who a few months ago implemented > Infinispan's Spring support and started to work on ISPN-78 - Large > Object Support. Due to very personal reasons and a job change that > required my undivided attention that latter endeavour went into a > hiatus. Sorry for that. > > Provided there's still interest in the community I'd like to continue > working on ISPN-78. But first let's see where we stand today. What > follows is in large part a reiteration of what was already said a few > months ago, but not all of us are blessed with an infallible memory. > > At https://github.com/obergner/infinispan/tree/t_ispn78 you will find > a very simple solution based on the concept of a StreamingHandler, to > be obtained via Cache.getStreamingHandler(): > > public interface StreamingHandler<K> { > void writeToKey(K key, InputStream largeObject); > OutputStream writeToKey(K key); > InputStream readFromKey(K key); > boolean removeKey(K key); > StreamingHandler<K> ; withFlags(Flag... flags ); > } > > This solution basically works yet needs some love before it could be > left out into the wild. I think it's not too bad but in the interim I > have come to prefer a different solution I would like to start > working on, namely a dedicated StreamingCache that's completely > separate from the regular Cache. I guess you'd obtain a StreamingCache from the CacheManager rather than obtaining it from a Cache?

> > Pros: > > 1. I think that it's highly unlikely that users would want store > large objects as well as "regular" objects within the same cache > instance. In my experience these two classes of objects are treated > entirely differently on the application level. > > 2. Moreover I *suspect* - though I know next to nothing about these > matters - that a user would have a hard time finding a set of tuning > parameters to satisfy the needs of both regular as well as large objects. > > 3. Keeping large object support as part of the regular cache *might* > entail that the same code paths would have to be optimized for both > regular as well as large objects. This *could* prove difficult in the > long term. > > 4. A separate StreamingCache would open up the possibility of > defining default settings tailored to the needs of large objects. > Furthermore I *suspect* that many cache settings would be irrelevant > for large objects. > > 5. The semantics of regular and large object caches are sufficiently > different to warrant a clean separation. Large objects will very > likely not be replicated across different nodes, and I *suspect* that > we will have a hard time supporting full-blown transactions for them. > > Cons: > > 1. Increased complexity. > > 2. Increased maintenance burden. Well, maybe. On the one hand, > there's one more kind of cache to take care of. On the other hand the > regular cache wouldn't have to pay attention to the streaming cache's > needs. > > 3. Higher implementation effort ;-) > > So my proposed course of action is: > > 1. Introduce a new interface StreamingCache that looks exactly like > the StreamingHandler above. > > 2. Rename StreamingHandlerImpl to StreamingCacheImpl. > > 3. Introduce a new concept, ChunkStore, a store for - you guessed it > - chunks. > > 4. Modify StreamingCacheImpl to delegate to ChunkStore instead of Cache. Why not create a fully flagged Cache under the hood and use that from the StreamingCacheImpl - this way you'd be able to reuse the current implementation. Same for the configuration, the StreamingCacheConfig would basically wrap an Configuration + some additional fields.

> > 5. And now for the fun part: create StreamingCacheConfiguration, a > basic, stripped down, bare bones, no frills variant of Configuration. > Whoa, that configuration code sure looks ... interesting. Could > probably need some pointers here. > > 6. Pray that I won't need a custom DataContainer and so forth. you might actually, as I imagine one might want to use eviction and passivation etc.. That would be solved with the cache approach I described previously.

> > What do you think? Does that make sense? Does it make basically > sense, yet you would suggest some improvements? Is it utter nonsense? +1 i.e. makes total sense to me.

> > The current implementation basically works and entails minimal > changes to the existing code base. The proposed solution is not > exactly rocket science, yet considerably more work. So I'd like to > make sure that I don't head in the wrong direction. _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev