[infinispan-dev] HotRod?? [ISPN-29 and a "custom protocol"]
Alex Kluge
java_kluge at yahoo.com
Wed Dec 9 15:21:37 EST 2009
> 1. Message IDs. Is this whap the client uses to
> match responses to requests sent, perhaps if sent in an
> async manner (NIO)? If so, is 4 bytes (a Java int)
> enough? Would we not want 8 bytes for this?
> Also, if it is unsigned, perhaps we could use a
> variable-length long?
In my experience, when used to match request/response pairs, this works
well when starting from 0 with each connection and if necessary wrapping.
It is doubtful that the client would have enough pending requests to
exceed this.
I have added more content to
http://www.jboss.org/community/wiki/RemoteCacheInteractions
And hopefully I can add material on the data encoding within a day, with examples soon thereafter.
> *Async ops (putAsync, removeAsync...etc)
For me, the communications protocol is asynchronous, where synchronous responses are manufactured by waiting on a future for the response.
> - keySet, entrySet, values: I don't plan to include them since they'd be
> very expensive both from a cache iteration perspective and from sending that stuff back to the client.
These would not be sent in the mainline usage of the cache, but they are very valuable for a management and measurement process. Since they would only be used where one is willing to incur the cost, it makes a lot of sense to provide them.
> Also, it might be wised to implement operations like
> replaceIfEquals using with a cas long, like memcached does
Yes, this is one of the big things it makes sense to add (I've seen cases in my code where it would be useful).
> Finally, memcached has this notion of getq where, as far as I can
> understand the spec, it's a quiet operation where the server holds up
> the return until you send a non quiet operation, i.e. a get or a noop.
Exactly. However, the utility of this is questionable as we have an asynchronous protocol, and can do the accumulation of the results on the client side.
Alex
--- On Wed, 12/9/09, Manik Surtani <manik at jboss.org> wrote:
> From: Manik Surtani <manik at jboss.org>
> Subject: Re: [infinispan-dev] HotRod?? [ISPN-29 and a "custom protocol"]
> To: "infinispan -Dev List" <infinispan-dev at lists.jboss.org>
> Date: Wednesday, December 9, 2009, 6:44 AM
> A few more comments:
>
> 1. Message IDs. Is this whap the client uses to
> match responses to requests sent, perhaps if sent in an
> async manner (NIO)? If so, is 4 bytes (a Java int)
> enough? Would we not want 8 bytes for this?
> Also, if it is unsigned, perhaps we could use a
> variable-length long?
>
> http://lucene.apache.org/java/2_9_1/fileformats.html#VInt
>
> Or maybe there is a better way to express this -
> UUIDs? Or is that too long/verbose?
>
> 2. Responses. How do you express hostname/IP
> (intelligence = 1) and hashes (intelligence = 2)? I
> presume the former is an IP represented as a String so that
> the client can open a connection to the node, and the latter
> is a 32-bit int since these are the hashes used by the
> ConsistentHash impl on the server nodes?
>
> 3. Responses. Number of owners - 4 bytes for
> this? Surely this is wrong, since you only have 2
> bytes for number of cluster members? :)
> Personally I think you could use 2 bytes for the former and
> a variable-length int/long for the latter (see above).
>
> Cheers
> Manik
>
> On 8 Dec 2009, at 19:36, Galder Zamarreno wrote:
>
> > Hi all,
> >
> > I've updated with a very early draft of the hot rod
> protocol:
> > http://www.jboss.org/community/wiki/HotRodProtocol
> >
> > As a reference, I've been using
> > http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol
> to see
> > what memcached did. Alex, as I emailed you yesterday,
> it'd be
> > interesting to see what you did too.
> >
> > First, several operations are still missing from the
> spec: remove,
> > removeIfPresent, *Async ops (putAsync,
> removeAsync...etc), putAll,
> > putForExternalRead, evict, version, name (get cache's
> name).
> >
> > Also, I still need to think how to add flags into each
> operation sent
> > across the network.
> >
> > There're some other operations that I'm consider
> whether to include or
> > not. Here's what I'm thinking at the moment:
> >
> > - keySet, entrySet, values: I don't plan to include
> them since they'd be
> > very expensive both from a cache iteration perspective
> and from sending
> > that stuff back to the client.
> >
> > - containsKey: I don't plan to include it, you can
> figure it out with
> > get command.
> >
> > - containsValue: No plan to include it since we don't
> support it.
> >
> > - size, isEmpty: Maybe include them as part of stats
> command like
> > memcached does?
> >
> > Also, it might be wised to implement operations like
> replaceIfEquals
> > using with a cas long, like memcached does
> > (http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol),
> rather
> > than sending the value cos that would be more network
> efficient. The old
> > value could be much more longer.
> >
> > Another thing to note here is that for expiry and
> maxIdle, I followed
> > the Java TimeUnit based definition. To make it easier
> for client
> > implementers, we could follow the memcached style,
> where seconds are
> > used always and if the number of seconds is bigger
> than 30 days, those
> > seconds represent UNIX time (number of seconds since
> 1st January 1970).
> > WDYT?
> >
> > Finally, memcached has this notion of getq where, as
> far as I can
> > understand the spec, it's a quiet operation where the
> server holds up
> > the return until you send a non quiet operation, i.e.
> a get or a noop. I
> > need to look further into it to understand how exactly
> works, but it'd
> > be interesting to know the thoughts of anyone who's
> dealt with this already.
> >
> > Cheers,
> >
> > On 12/03/2009 06:22 PM, Manik Surtani wrote:
> >> Great, thanks Alex!
> >>
> >> On 3 Dec 2009, at 05:00, Alex Kluge wrote:
> >>
> >>> Hi,
> >>>
> >>>>> I'll see what I can do about
> documenting it...
> >>>>
> >>>> Yeah lets do that - I would recommend
> using the JBoss.org
> >>>> wiki pages so that the designs can be
> distilled by all of us
> >>>> and then linked to from the other
> Infinispan design pages.
> >>>
> >>> I've started this at
> >>>
> >>> http://www.jboss.org/community/wiki/RemoteCacheInteractions
> >>>
> >>> I will start with a description of what I
> have, which is a good multilevel cache implementation. Then
> get to a description of some of the additional motivations
> and requirements for a data grid, which is a bit more
> general, and use that to motivate changes in the protocol.
> >>>
> >>>
>
>
> Alex
> >>>
> >>>
> >>>
> >>>
> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>
> >> --
> >> Manik Surtani
> >> manik at jboss.org
> >> Lead, Infinispan
> >> Lead, JBoss Cache
> >> http://www.infinispan.org
> >> http://www.jbosscache.org
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> > --
> > Galder Zamarreño
> > Sr. Software Engineer
> > Infinispan, JBoss Cache
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
More information about the infinispan-dev
mailing list