[infinispan-dev] chunking ability on the JDBC cacheloader

["이희승 (Trustin Lee)"]

On 05/20/2011 05:01 PM, Sanne Grinovero wrote:
> 2011/5/20 Manik Surtani<manik at jboss.org>:
>> Is spanning rows the only real solution?  As you say it would mandate using transactions to keep multiple rows coherent, and 'm not sure if everyone would want to enable transactions for this.
>
> I think that using multiple rows for variable sized data is a good
> approach, unless the database supports very variable blob sizes.
>
> I'm not suggesting that we should heavily fragment each value, more
> the opposite I think that as right now people should define a
> reasonable estimate to keep it in one column, but the JDBC cacheloader
> should be able to handle the case in which the estimate is wrong and
> the blob can't be stored completely.
>
> As Trustin said, ISPN-701 could be related so that exact
> implementation depends on database capabilities.

Yeah, let's post the relevant ideas to the JIRA page for easier tracking.

>>
>> On 19 May 2011, at 19:06, Sanne Grinovero wrote:
>>
>>> As mentioned on the user forum [1], people setting up a JDBC
>>> cacheloader need to be able to define the size of columns to be used.
>>> The Lucene Directory has a feature to autonomously chunk the segment
>>> contents at a configurable specified byte number, and so has the
>>> GridFS; still there are other metadata objects which Lucene currently
>>> doesn't chunk as it's "fairly small" (but undefined and possibly
>>> growing), and in a more general sense anybody using the JDBC
>>> cacheloader would face the same problem: what's the dimension I need
>>> to use ?
>>>
>>> While in most cases the maximum size can be estimated, this is still
>>> not good enough, as when you're wrong the byte array might get
>>> truncated, so I think the CacheLoader should take care of this.
>>>
>>> what would you think of:
>>> - adding a max_chunk_size option to JdbcStringBasedCacheStoreConfig
>>> and JdbcBinaryCacheStore
>>> - have them store in multiple rows the values which would be bigger
>>> than max_chunk_size
>>> - this will need transactions, which are currently not being used by
>>> the cacheloaders
>>>
>>> It looks like to me that only the JDBC cacheloader has these issues,
>>> as the other stores I'm aware of are more "blob oriented". Could it be
>>> worth to build this abstraction in an higher level instead of in the
>>> JDBC cacheloader?
>>>
>>> Cheers,
>>> Sanne
>>>
>>> [1] - http://community.jboss.org/thread/166760
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Manik Surtani
>> manik at jboss.org
>> twitter.com/maniksurtani
>>
>> Lead, Infinispan
>> http://www.infinispan.org
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Trustin Lee, http://gleamynode.net/