[infinispan-dev] TwoWayKey2StringMapper and keys which are never stored

[Sanne Grinovero]

2010/8/12 Mircea Markus <mircea.markus at jboss.com>:
> On 12 Aug 2010, at 10:33, Sanne Grinovero wrote:
>
>> Hello,
>> While implementing a TwoWayKey2StringMapper for the Lucene Directory I
>> noticed that
>> it's mandatory to implement two-way conversions also for keys which
>> I'm not going to ever store.
>>
>> This directory has some different keys types, some need to be
>> eventually storable (if a Store is configured),
>> while some others are always handled with the Flag.SKIP_CACHE_STORE
>> and should never be stored.
>>
>> Still if I don't implement conversion of this type, I get a stacktrace
>> about this type being not supported by
>> the LuceneKey2StringMapper during rehashing.
>> I'm not able to grasp the reason for this two-way key conversion to be
>> needed for rehashing - I can only guess that you have good reasons for
>> that - but I'm totally lost at why this implementation is being called
>> for keys which I actually want to make sure they are never stored.
> Let me try to explain how even the keys that are not persisted(SKIP_CACHE_STORE) get involved in rehashing.
> Initial cluster {A,B}
> C joins; C asks both A and B to supply the state that should be migrated to C (rehashing)
> Both A and B do the following:
> 1.iterate over the in-memory state and collect the entries that should be moved to A
> 2. iterate over all the keys  in the cache store (but the in-memory ones), and again it collects all the entries that should be moved to A
> All the  entries collected in 1. and 2. are send back to A and which integrates them
> A receives the entries  and writes them both in memory and in the cache store. Here is where you receive an exception: A doesn't know (and cannot know) which entries were not to be persisted(SKIP_CACHE_STORE), but it persists everything.
>>
>> Could this limitation be removed, or am I mandated to use two
>> differently configured caches to store this information?
> This would be a good solution: otherwise keys that you didn't want to be persisted (for a reason) would get persisted after rehashing.

Thank you very much, so I finally understand and discover that there's
no safe way to prevent a specific key to ever hit the Store.
I'm afraid that I'll have to change the InfinispanDirectory constructor from

InfinispanDirectory(Cache cache, String indexName, LockFactory lf, int
chunkSize) {

to

InfinispanDirectory(Cache dataCache, Cache metaDataCache, Cache
locksCache, String indexName, LockFactory lf, int chunkSize) {

so that we can recommend this setup:
* dataCache con use DIST [+Store] [+L1 cache]
* metaDataCache REPL [+Store]
* locksCache REPL

this was something I was planning to propose anyway, but thought to do
it in next release and optionally,
but I see now that it's an urgent requirement.

Cheers,
Sanne