[infinispan-dev] chunking ability on the JDBC cacheloader

[Sanne Grinovero]

2011/5/20 "이희승 (Trustin Lee)" <trustin at gmail.com>:
> On 05/20/2011 03:06 AM, Sanne Grinovero wrote:
>> As mentioned on the user forum [1], people setting up a JDBC
>> cacheloader need to be able to define the size of columns to be used.
>> The Lucene Directory has a feature to autonomously chunk the segment
>> contents at a configurable specified byte number, and so has the
>> GridFS; still there are other metadata objects which Lucene currently
>> doesn't chunk as it's "fairly small" (but undefined and possibly
>> growing), and in a more general sense anybody using the JDBC
>> cacheloader would face the same problem: what's the dimension I need
>> to use ?
>>
>> While in most cases the maximum size can be estimated, this is still
>> not good enough, as when you're wrong the byte array might get
>> truncated, so I think the CacheLoader should take care of this.
>>
>> what would you think of:
>>  - adding a max_chunk_size option to JdbcStringBasedCacheStoreConfig
>> and JdbcBinaryCacheStore
>>  - have them store in multiple rows the values which would be bigger
>> than max_chunk_size
>>  - this will need transactions, which are currently not being used by
>> the cacheloaders
>>
>> It looks like to me that only the JDBC cacheloader has these issues,
>> as the other stores I'm aware of are more "blob oriented". Could it be
>> worth to build this abstraction in an higher level instead of in the
>> JDBC cacheloader?
>
> I'm not sure if I understand the idea correctly.  Do you mean an entry
> should span to more than one row if the entry's value is larger than the
> maximum column capacity?  I guess it's not about keys, right?

yes, having it on the keys would be ideal (just as a safety feature)
but I don't think we could map that.
table schema proposal would look like:

[serialized key] [chunk id] [timestamp] [blob or part of it]

instead of
[serialized key] [timestamp] [blob]

>
> Sounds like a good idea for ISPN-701 because it will surely result in
> different schema.
>
>> [1] - http://community.jboss.org/thread/166760
>
> --
> Trustin Lee, http://gleamynode.net/
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>