[infinispan-dev] HotRod?? [ISPN-29 and a "custom protocol"]

Galder Zamarreno galder at redhat.com
Thu Dec 10 07:54:33 EST 2009 


On 12/09/2009 09:21 PM, Alex Kluge wrote:
>> 1.  Message IDs.  Is this whap the client uses to
>> match responses to requests sent, perhaps if sent in an
>> async manner (NIO)?  If so, is 4 bytes (a Java int)
>> enough?  Would we not want 8 bytes for this?
>> Also, if it is unsigned, perhaps we could use a
>> variable-length long?

Yeah, the idea is to use it to match request with response.

>
> In my experience, when used to match request/response pairs, this works
> well when starting from 0 with each connection and if necessary wrapping.
> It is doubtful that the client would have enough pending requests to
> exceed this.

I agree with Manik that variable length long would be a good solution, 
compact and expandable if necessary.

>
> I have added more content to
>   http://www.jboss.org/community/wiki/RemoteCacheInteractions
>
> And hopefully I can add material on the data encoding within a day, with examples soon thereafter.
>
>> *Async ops (putAsync, removeAsync...etc)
> For me, the communications protocol is asynchronous, where synchronous responses are manufactured by waiting on a future for the response.

True, good point. This is a client side decision.

>
>>   - keySet, entrySet, values: I don't plan to include them since they'd be
>>   very expensive both from a cache iteration perspective and from sending that stuff back to the client.
>
>   These would not be sent in the mainline usage of the cache, but they are very valuable for a management and measurement process. Since they would only be used where one is willing to incur the cost, it makes a lot of sense to provide them.

For management and measurement, I'd suggest using Jopr, jconsole or 
similar tools. Also, once we put those ops out there, who knows when 
they'd be used. We can't control whether it's mainline or not :)

>
>> Also, it might be wised to implement operations like
>> replaceIfEquals  using with a cas long, like memcached does
>
> Yes, this is one of the big things it makes sense to add (I've seen cases in my code where it would be useful).

+1

>
>> Finally, memcached has this notion of getq where, as far as I can
>> understand the spec, it's a quiet operation where the server holds up
>> the return until you send a non quiet operation, i.e. a get or a noop.
>
> Exactly. However, the utility of this is questionable as we have an asynchronous protocol, and can do the accumulation of the results on the client side.

 From what I understand, this is more about server sending up the 
responses all in one response, more about network utilization rather 
doing accumulation on the client side itself.

I don't see them as top priority though, so most likely I'll leave them 
out. We can always add them at a later stage if we think it's worth 
implementing them.

>
>                                      Alex
>
> --- On Wed, 12/9/09, Manik Surtani<manik at jboss.org>  wrote:
>
>> From: Manik Surtani<manik at jboss.org>
>> Subject: Re: [infinispan-dev] HotRod?? [ISPN-29 and a "custom protocol"]
>> To: "infinispan -Dev List"<infinispan-dev at lists.jboss.org>
>> Date: Wednesday, December 9, 2009, 6:44 AM
>> A few more comments:
>>
>> 1.  Message IDs.  Is this whap the client uses to
>> match responses to requests sent, perhaps if sent in an
>> async manner (NIO)?  If so, is 4 bytes (a Java int)
>> enough?  Would we not want 8 bytes for this?
>> Also, if it is unsigned, perhaps we could use a
>> variable-length long?
>>
>>      http://lucene.apache.org/java/2_9_1/fileformats.html#VInt
>>
>> Or maybe there is a better way to express this -
>> UUIDs?  Or is that too long/verbose?
>>
>> 2.  Responses.  How do you express hostname/IP
>> (intelligence = 1) and hashes (intelligence = 2)?  I
>> presume the former is an IP represented as a String so that
>> the client can open a connection to the node, and the latter
>> is a 32-bit int since these are the hashes used by the
>> ConsistentHash impl on the server nodes?
>>
>> 3.  Responses.  Number of owners - 4 bytes for
>> this?  Surely this is wrong, since you only have 2
>> bytes for number of cluster members?  :)
>> Personally I think you could use 2 bytes for the former and
>> a variable-length int/long for the latter (see above).
>>
>> Cheers
>> Manik
>>
>> On 8 Dec 2009, at 19:36, Galder Zamarreno wrote:
>>
>>> Hi all,
>>>
>>> I've updated with a very early draft of the hot rod
>> protocol:
>>> http://www.jboss.org/community/wiki/HotRodProtocol
>>>
>>> As a reference, I've been using
>>> http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol
>> to see
>>> what memcached did. Alex, as I emailed you yesterday,
>> it'd be
>>> interesting to see what you did too.
>>>
>>> First, several operations are still missing from the
>> spec: remove,
>>> removeIfPresent, *Async ops (putAsync,
>> removeAsync...etc), putAll,
>>> putForExternalRead, evict, version, name (get cache's
>> name).
>>>
>>> Also, I still need to think how to add flags into each
>> operation sent
>>> across the network.
>>>
>>> There're some other operations that I'm consider
>> whether to include or
>>> not. Here's what I'm thinking at the moment:
>>>
>>> - keySet, entrySet, values: I don't plan to include
>> them since they'd be
>>> very expensive both from a cache iteration perspective
>> and from sending
>>> that stuff back to the client.
>>>
>>> - containsKey: I don't plan to include it, you can
>> figure it out with
>>> get command.
>>>
>>> - containsValue: No plan to include it since we don't
>> support it.
>>>
>>> - size, isEmpty: Maybe include them as part of stats
>> command like
>>> memcached does?
>>>
>>> Also, it might be wised to implement operations like
>> replaceIfEquals
>>> using with a cas long, like memcached does
>>> (http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol),
>> rather
>>> than sending the value cos that would be more network
>> efficient. The old
>>> value could be much more longer.
>>>
>>> Another thing to note here is that for expiry and
>> maxIdle, I followed
>>> the Java TimeUnit based definition. To make it easier
>> for client
>>> implementers, we could follow the memcached style,
>> where seconds are
>>> used always and if the number of seconds is bigger
>> than 30 days, those
>>> seconds represent UNIX time (number of seconds since
>> 1st January 1970).
>>> WDYT?
>>>
>>> Finally, memcached has this notion of getq where, as
>> far as I can
>>> understand the spec, it's a quiet operation where the
>> server holds up
>>> the return until you send a non quiet operation, i.e.
>> a get or a noop. I
>>> need to look further into it to understand how exactly
>> works, but it'd
>>> be interesting to know the thoughts of anyone who's
>> dealt with this already.
>>>
>>> Cheers,
>>>
>>> On 12/03/2009 06:22 PM, Manik Surtani wrote:
>>>> Great, thanks Alex!
>>>>
>>>> On 3 Dec 2009, at 05:00, Alex Kluge wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>>>> I'll see what I can do about
>> documenting it...
>>>>>>
>>>>>> Yeah lets do that - I would recommend
>> using the JBoss.org
>>>>>> wiki pages so that the designs can be
>> distilled by all of us
>>>>>> and then linked to from the other
>> Infinispan design pages.
>>>>>
>>>>> I've started this at
>>>>>
>>>>>    http://www.jboss.org/community/wiki/RemoteCacheInteractions
>>>>>
>>>>> I will start with a description of what I
>> have, which is a good multilevel cache implementation. Then
>> get to a description of some of the additional motivations
>> and requirements for a data grid, which is a bit more
>> general, and use that to motivate changes in the protocol.
>>>>>
>>>>>
>>
>>
>>             Alex
>>>>>
>>>>>
>>>>>
>>>>>
>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> --
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> Lead, Infinispan
>>>> Lead, JBoss Cache
>>>> http://www.infinispan.org
>>>> http://www.jbosscache.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>
>>> --
>>> Galder Zamarreño
>>> Sr. Software Engineer
>>> Infinispan, JBoss Cache
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Manik Surtani
>> manik at jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>> http://www.infinispan.org
>> http://www.jbosscache.org
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache

More information about the infinispan-dev mailing list