[infinispan-issues] [JBoss JIRA] Commented: (ISPN-78) Large object support

[ https://issues.jboss.org/browse/ISPN-78?page=com.atlassian.jira.plugin.sy... ] Olaf Bergner commented on ISPN-78: ---------------------------------- Hi Trustin, 1) Well, chunking is the solution MongoDB found for their GridFS - http://www.mongodb.org/display/DOCS/GridFS+Specification - and so far it seems to serve them well. At least I know that people are successfully using GridFS. That being said it might indeed be desirable to additionally support an alternative where large objects are written through the cache to disk. I personally would opt to first implement the current solution based on chunking and see how it goes. Later, if the need arises, we should be able to transparently support write-through caches, too. 2) Right now the interface is {code} public interface StreamingHandler<K> { void writeToKey(K key, InputStream largeObject); OutputStream writeToKey(K key); InputStream readFromKey(K key); boolean removeKey(K key); StreamingHandler<K> withFlags(Flag... flags); } {code} i.e. I support {{transferFrom(InputStream)}} via {{writeToKey(K, InputStream}}} and {{write(byte)}} via {{OutputStream writeToKey(K)}}. Using the latter a client will have to call {{close()}} on the {{OutputStream}} returned to complete the write. It seems to me that this very similar to your {{LargeObjectWriter}}. 3) Yes, method naming could definitely be improved and {{newLargeObject(K key)}} seems to make sense. I'm not so sure, though, that it would still be appropriate when *overwriting* an existing key. Maybe simply {{largeObject(K key)}}? 4) I don't think a user should know about the chunks a large object is partitioned into. After all, it's just an implementation detail and the exact structure of the partitioning scheme chosen by INFINISPAN is likely immaterial to the domain at hand. One exception might be when the user chooses the configurable chunk size according to not technical but functional reasons, i.e. he knows that a single record within a large file will *always* be 256k in size. But then he may use the {{InputStream}} returned from {{readKey(K key)}} to read a large object in chunks of arbitrary size. As a general approach I would like to first get the interface and - by extension - implementation details directly affecting the interface right. How to best implement that interface is a question we may answer later. The more I think about it, the less likely it seems to me that a user will *ever* want to mix "regular" and large objects within a single cache. At least most applications I know of wouldn't want to do so. Plus I imagine that it might prove difficult to tune a cache containing regular as well as large objects. Maybe we should restrict storage of large objects to dedicated caches? This would have the added bonus of being able to define appropriate default settings for large object caches. Any thoughts? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira