[infinispan-dev] API behaviour when using DIST

Tue Apr 21 09:25:00 EDT 2009

On 21 Apr 2009, at 14:08, Bela Ban wrote:

> Can't we reuse cacheMode (REPL_SYNC, REPL_ASYNC) ?

Those were deprecated in JBC 2.x.  :-)  From JBC3, the cache mode  
(LOCAL, REPL, INVAL) and the transport mode (SYNC vs ASYNC) are  
separated in config.  Internally, they have always been stored  
separately.

>
>
> Manik Surtani wrote:
>> And in terms of configuration - this is how I see it looking:
>>
>> <!-- This will allow you to use ASYNC with DIST -->
>> <!-- If used with SYNC + DIST, it will optimise away the remote get  
>> before a write -->
>> <unsafe unreliableReturnValues="true" />
>>
>> <cluster mode="dist">
>>  <async />
>> </cluster>
>>
>> Reckon this is clear and intuitive enough for users?
>>
>> Cheers
>> Manik
>>
>>
>> On 21 Apr 2009, at 11:41, Manik Surtani wrote:
>>
>>> Ok, after discussing this on IRC and the clustering conf call,  
>>> here is what we will do:
>>>
>>> 1.  Non-transactional
>>>
>>> * Only support DIST in SYNC mode.  Throw a configuration exception  
>>> if DIST and ASYNC are detected.
>>> * Allow DIST_ASYNC if an unsafe flag is set to true  
>>> (breakApi=true?) in the config, so that the user is explicitly  
>>> aware that API return values will be unreliable.
>>> * Make sure this is documented in the Javadocs as well as the FAQs
>>>
>>> 2.  Transactional - Async
>>>
>>> * As above, do not support unless breakApi = true.  In which case  
>>> this is simple - no return values, only broadcast modifications in  
>>> prepare as we do with REPL_SYNC.
>>>
>>> 3.  Transactional - Sync
>>>
>>> * Offer 2 options here.  Option 1 is simple and we already have  
>>> this.  Option 2 will need to be written.
>>>
>>> Option 1:  break API.  This will behave the same as async, in that  
>>> we don't care about return values, but we still use a 2-phase  
>>> commit protocol for the transaction boundary.  Users will still  
>>> need to specify breakApi = true so this is explicit.
>>>
>>> Option 2: Before any write RPC call is broadcast, a remote get is  
>>> performed.  This pulls back the necessary values so that a  
>>> reliable return value is available.
>>>
>>> Cheers
>>> Manik
>>>
>>> On 17 Apr 2009, at 12:18, Manik Surtani wrote:
>>>
>>>> This gets even more interesting when we have transactions in  
>>>> play.  E.g.:
>>>>
>>>> K resides on {A, B} in a cluster of {A, B, C}
>>>>
>>>> On C, we do:
>>>>
>>>> 1.  tx.begin()
>>>> 2.  retval = C.replace(K, V, V2)
>>>> 3.  // do something with retval
>>>> 4.  tx.commit()
>>>>
>>>> It is one thing to set the expectation of unreliable return  
>>>> values for async mode as defined below, but the above would be  
>>>> tricky even in the case of sync mode.  On line 2 we would have to  
>>>> start the tx on remote nodes and start processing the commands  
>>>> and pull back results.  And on 4, the prepare logic would have to  
>>>> change to assume that all writes have already been executed.
>>>>
>>>> On the plus side, this means we have txs that will fail fast(er)  
>>>> if remote locks cannot be be acquired...
>>>>
>>>> Thoughts and comments?
>>>>
>>>> - Manik
>>>>
>>>> On 16 Apr 2009, at 14:33, Manik Surtani wrote:
>>>>
>>>>> Some of the API methods' return values may not be as expected  
>>>>> when using DIST.  Just wanted to run this by you guys to see  
>>>>> what you thought.
>>>>>
>>>>> Specifically, what I am referring to is methods like put(),  
>>>>> putIfAbsent(), and the 2 versions of remove() and replace().
>>>>>
>>>>> Now when these are executed in any other cache mode (LOCAL,  
>>>>> REPL, INVAL) return values are as expected from their  
>>>>> java.util.Map contracts.
>>>>>
>>>>> However with DIST, there are 2 separate ways this could go.
>>>>>
>>>>> 1.  The key in question is mapped to the local cache on which  
>>>>> the method is invoked.
>>>>>
>>>>> This case behaves normally as well as per the Map contract.
>>>>>
>>>>> 2.  The key in question is *not* mapped to the local cache.   
>>>>> E.g., K is mapped to nodes A and B, and on C we do C.remove().
>>>>>
>>>>> I can stick with the map contract if each of these methods first  
>>>>> does a remote lookup of the key and stores this in L1 (or even  
>>>>> in the invocation context and then throw it away if there is no  
>>>>> L1) but this is unnecessarily wasteful IMO.
>>>>>
>>>>> The other approach is to realise the command pertains to a key  
>>>>> for which the local cache is not the owner, and replicate the  
>>>>> call to the remote nodes.
>>>>>
>>>>> What happens next is also a matter of tuning:  we could
>>>>>
>>>>> a) replicate + ignore responses.  Then the retval to these  
>>>>> methods are indeterminate, and probably unusable.
>>>>> b) replicate + wait for response.  This will adhere to the Map  
>>>>> contract, albeit at a cost.  Probably not possible if we use  
>>>>> DIST_ASYNC.
>>>>> c) L1: we could either "correct" the L1 cache with the change,  
>>>>> or remove the key forcing another lookup for the next time around.
>>>>>
>>>>> What do people think?  Maybe offer all strategies?  Or will this  
>>>>> then become configuration hell?  :)
>>>>>
>>>>> Cheers
>>>>> -- 
>>>>> Manik Surtani
>>>>> manik at jboss.org
>>>>> Lead, Infinispan
>>>>> Lead, JBoss Cache
>>>>> http://www.infinispan.org
>>>>> http://www.jbosscache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> -- 
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> Lead, Infinispan
>>>> Lead, JBoss Cache
>>>> http://www.infinispan.org
>>>> http://www.jbosscache.org
>>>>
>>>>
>>>>
>>>>
>>>
>>> -- 
>>> Manik Surtani
>>> manik at jboss.org
>>> Lead, Infinispan
>>> Lead, JBoss Cache
>>> http://www.infinispan.org
>>> http://www.jbosscache.org
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> -- 
>> Manik Surtani
>> manik at jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>> http://www.infinispan.org
>> http://www.jbosscache.org
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> -- 
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss - a division of Red Hat
>

--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org