1. Message IDs. Is this whap the client uses to
match responses to requests sent, perhaps if sent in an
async manner (NIO)? If so, is 4 bytes (a Java int)
enough? Would we not want 8 bytes for this?
Also, if it is unsigned, perhaps we could use a
variable-length long?
In my experience, when used to match request/response pairs, this works
well when starting from 0 with each connection and if necessary wrapping.
It is doubtful that the client would have enough pending requests to
exceed this.
I have added more content to
http://www.jboss.org/community/wiki/RemoteCacheInteractions
And hopefully I can add material on the data encoding within a day, with examples soon
thereafter.
*Async ops (putAsync, removeAsync...etc)
For me, the
communications protocol is asynchronous, where synchronous responses are manufactured by
waiting on a future for the response.
- keySet, entrySet, values: I don't plan to include them since
they'd be
very expensive both from a cache iteration perspective and from sending that stuff back
to the client.
These would not be sent in the mainline usage of the cache, but they are very valuable
for a management and measurement process. Since they would only be used where one is
willing to incur the cost, it makes a lot of sense to provide them.
Also, it might be wised to implement operations like
replaceIfEquals using with a cas long, like memcached does
Yes, this is one of the big things it makes sense to add (I've seen cases in my code
where it would be useful).
Finally, memcached has this notion of getq where, as far as I can
understand the spec, it's a quiet operation where the server holds up
the return until you send a non quiet operation, i.e. a get or a noop.
Exactly. However, the utility of this is questionable as we have an asynchronous protocol,
and can do the accumulation of the results on the client side.
Alex
--- On Wed, 12/9/09, Manik Surtani <manik(a)jboss.org> wrote:
From: Manik Surtani <manik(a)jboss.org>
Subject: Re: [infinispan-dev] HotRod?? [ISPN-29 and a "custom protocol"]
To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org>
Date: Wednesday, December 9, 2009, 6:44 AM
A few more comments:
1. Message IDs. Is this whap the client uses to
match responses to requests sent, perhaps if sent in an
async manner (NIO)? If so, is 4 bytes (a Java int)
enough? Would we not want 8 bytes for this?
Also, if it is unsigned, perhaps we could use a
variable-length long?
http://lucene.apache.org/java/2_9_1/fileformats.html#VInt
Or maybe there is a better way to express this -
UUIDs? Or is that too long/verbose?
2. Responses. How do you express hostname/IP
(intelligence = 1) and hashes (intelligence = 2)? I
presume the former is an IP represented as a String so that
the client can open a connection to the node, and the latter
is a 32-bit int since these are the hashes used by the
ConsistentHash impl on the server nodes?
3. Responses. Number of owners - 4 bytes for
this? Surely this is wrong, since you only have 2
bytes for number of cluster members? :)
Personally I think you could use 2 bytes for the former and
a variable-length int/long for the latter (see above).
Cheers
Manik
On 8 Dec 2009, at 19:36, Galder Zamarreno wrote:
> Hi all,
>
> I've updated with a very early draft of the hot rod
protocol:
>
http://www.jboss.org/community/wiki/HotRodProtocol
>
> As a reference, I've been using
>
http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol
to see
> what memcached did. Alex, as I emailed you yesterday,
it'd be
> interesting to see what you did too.
>
> First, several operations are still missing from the
spec: remove,
> removeIfPresent, *Async ops (putAsync,
removeAsync...etc), putAll,
> putForExternalRead, evict, version, name (get cache's
name).
>
> Also, I still need to think how to add flags into each
operation sent
> across the network.
>
> There're some other operations that I'm consider
whether to include or
> not. Here's what I'm thinking at the moment:
>
> - keySet, entrySet, values: I don't plan to include
them since they'd be
> very expensive both from a cache iteration perspective
and from sending
> that stuff back to the client.
>
> - containsKey: I don't plan to include it, you can
figure it out with
> get command.
>
> - containsValue: No plan to include it since we don't
support it.
>
> - size, isEmpty: Maybe include them as part of stats
command like
> memcached does?
>
> Also, it might be wised to implement operations like
replaceIfEquals
> using with a cas long, like memcached does
> (
http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol),
rather
> than sending the value cos that would be more network
efficient. The old
> value could be much more longer.
>
> Another thing to note here is that for expiry and
maxIdle, I followed
> the Java TimeUnit based definition. To make it easier
for client
> implementers, we could follow the memcached style,
where seconds are
> used always and if the number of seconds is bigger
than 30 days, those
> seconds represent UNIX time (number of seconds since
1st January 1970).
> WDYT?
>
> Finally, memcached has this notion of getq where, as
far as I can
> understand the spec, it's a quiet operation where the
server holds up
> the return until you send a non quiet operation, i.e.
a get or a noop. I
> need to look further into it to understand how exactly
works, but it'd
> be interesting to know the thoughts of anyone who's
dealt with this already.
>
> Cheers,
>
> On 12/03/2009 06:22 PM, Manik Surtani wrote:
>> Great, thanks Alex!
>>
>> On 3 Dec 2009, at 05:00, Alex Kluge wrote:
>>
>>> Hi,
>>>
>>>>> I'll see what I can do about
documenting it...
>>>>
>>>> Yeah lets do that - I would recommend
using the
JBoss.org
>>>> wiki pages so that the designs can be
distilled by all of us
>>>> and then linked to from the other
Infinispan design pages.
>>>
>>> I've started this at
>>>
>>>
http://www.jboss.org/community/wiki/RemoteCacheInteractions
>>>
>>> I will start with a description of what I
have, which is a good multilevel cache implementation. Then
get to a description of some of the additional motivations
and requirements for a data grid, which is a bit more
general, and use that to motivate changes in the protocol.
>>>
>>>
Alex
>>>
>>>
>>>
>>>
_______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev(a)lists.jboss.org
>>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Manik Surtani
>> manik(a)jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>>
http://www.infinispan.org
>>
http://www.jbosscache.org
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Manik Surtani
manik(a)jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev