[infinispan-dev] HotRod?? [ISPN-29 and a "custom protocol"]

[Galder Zamarreno]

On 12/10/2009 03:15 PM, Manik Surtani wrote:
>
> On 10 Dec 2009, at 12:23, Galder Zamarreno wrote:
>>>> Also, it might be wised to implement operations like replaceIfEquals
>>>> using with a cas long, like memcached does
>>>> (http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol), rather
>>>> than sending the value cos that would be more network efficient. The old
>>>> value could be much more longer.
>>>
>>> Not sure I understand.  So we have 2 replace methods (thanks to ConcurrentMap) - replace(k, v) which succeeds if k is mapped to something, and replace(k, v1, v2) which succeeds if k is mapped to v1.  From what I see, only the former is supported by memcached's binary protocol and not the latter.  Are you suggesting supporting the latter in HotRod?
>>
>> I'm suggesting supporting both actually.
>>
>> replace(k, v) which succeeds if k is mapped to something - that's the
>> same as memcached's replace command. I plan to support that.
>>
>> replace(k, v1, v2) is what memcached does with 'cas' but instead of
>> passing the old value, it passes the cas id, which is 8 bytes at the
>> most. The old value could be a lot longer. In ConcurrentMap, replace
>> works by passing the old value.
>
> Sure, and that's in the same VM so you pass by reference.  Across the wire, this may involve serializing the entire (old) value which may be quite large.
>
>> I think, to save network space, I think we should implement it the way
>> memcached does. IOW, whenever something gets stored, it's stored with a
>> cas id and this can later be used to replace a value, only of the cas is
>> the one as we've passed.
>
> You get this cas id back as a return value?  Which you can then store and use in future 'replace' commands?  Is this it?

Yeah, u can get the cas id via gets command.

>
>> In the txt protocol, I implemented the cas id using System.nanoTime().
>
> Is this cas id meant to be unique?  If so, System.nanoTime() may not be ...

Not over time. You want to be unique from the minute you retrieved it 
until you use it and for operations on that particular key.

The cas Id is simply used for comparison, to check whether someone else 
has changed the entry since you retrieved it. System.nanoTime() gives 
you precisely that, anyone that modifies that entry in that machine will 
definitely have a different nano time. Also, since nano time is based 
off an arbitrary time, the chances that a different machine will produce 
the same nano time when modifying that very same key are almost very 
very small.

>
>>>> Another thing to note here is that for expiry and maxIdle, I followed
>>>> the Java TimeUnit based definition. To make it easier for client
>>>> implementers, we could follow the memcached style, where seconds are
>>>> used always and if the number of seconds is bigger than 30 days, those
>>>> seconds represent UNIX time (number of seconds since 1st January 1970).
>>>> WDYT?
>>>
>>> This is done to reduce bytes on the wire, since expressing something in millis often means extra bytes and often meaningless since network latency can be worth several hundred millis already.  So it makes sense for the protocol to express this in seconds.  Not sure I understand your comment re: seconds>   30 days.  Are you referring to the time being treated as relative to the UNIX epoch [1] as opposed to relative to the current time snapshot?
>>
>> Indeed, when you pass a number of seconds that is bigger than 30 days,
>> memcached understands it as the offset from 1/1/1970. Otherwise, if
>> below 30 days, it thinks it's relative to the current time.
>>
>> Good point about sending anything below seconds, it's wasteful and
>> netwrok latency probably invalidates. Assuming we only allowed, seconds,
>> minutes, hours, and days as time units, we'll be saving space compared
>> to memcached.
>>
>> I'm not sure of the frequency of hours/days as time units, so I'd
>> inclined to only support seconds and use the same algorithm as memcached
>> simply, as this reduces complexity of existing clients that might be
>> used to memcached protocol.
>
> Yes, the protocol should just use seconds.  Client libs on top could add syntactic sugar like TimeUnit.*, etc.
>
> Cheers
> --
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache