[infinispan-dev] Hot Rod - pt3
Galder Zamarreno
galder at redhat.com
Tue Jan 5 04:52:45 EST 2010
On 01/04/2010 10:44 PM, Alex Kluge wrote:
>>> - No events. I used events, that is asynchronous messages originating
>>> from the server when certain conditions are met, to notify clients
>>> that data was written to the server. This was mostly driven by the
>>> need to notify an L1 cache of a write to the cache server. It would
>>> be good to allow for this usage in this protocol. Note that this is
>>> another case where the op code is useful to have as part of the
>>> message.
>>
>> Isn't this expensive? Doesn't this mean the server
>> has to fire off messages to each and every connected client,
>> whether the clients are interested in these messages or
>> not?
>
> Not necessarily. There are a number of options. It wouldn't be
> difficult to allow the clients to register for the events, and
> then only send them to the interested clients. The events can
> be sent asynchronously (a separate thread), thus they won't
> delay the response to a write. Piratically speaking, they aren't
> that expensive.
I'm still not convinced by this either. What you're talking about sounds
like JMS to me :). Right now, the only situation where I can see this
being useful is for sending back cluster formation changes but we found
simpler way to deal with it and that doesn't require the added complexity.
My other source of doubt has already being highlighted in a previous
thread and relates to firewalls, which generally block persistent
connections.
It's unclear here whether in your case you had control over the clients
and the environment linking clients to your server and whether you had
any issues with the topics just mentioned.
I do find an asynchronous event model easier to extend in future if we
want to send other information back to the clients, and possibly easier
to filter client capabilities. You could assume that all clients must be
able to handle a response as it is, and keep things like cluster
formation changes as asynchronous events that more complex clients will
only be able to handle.
>
>>> - What happens if the key or the value is not text? I have a way of
>>> representing the data to allow for a wide variety of data types,
>>> even allowing for arrays or maps. This will make the protocol more
>>> complex, but the assumption that the data is a string is rather
>>> limiting. This is already sketched out in the wiki.
Hmmmmm, I don't think I've made any assumptions in the wiki that keys or
values are Strings unless I've made a mistake somewhere (maybe in the
example where I've used a particular encoding for Strings?). My thoughts
around this was that I was gonna treat them both as byte[]...
>>
>> There is some work around non-text data in the RESTful
>> cache server. Adding MIME types to any opaque data
>> allows us to attach handlers to this data, such as XML or
>> JSON data. What do you have in mind?
>
> The idea is to prefix each data block with a data type. This is a
> lightweight binary protocol, so a full fledged mime type would be
> overkill. There is a discussion and example in the Encoding Data
> section of this page:
>
> http://community.jboss.org/wiki/RemoteCacheInteractions
>
> Data types are limited to things like integer, byte, string, boolean, etc.
> Or, if it isn't a recognised type, the native platform serialisation is
> used. There can be arrays of these types, or maps as well.
>
> Each data type is represented by a bit, and they can be used in
> combinations. An array of bytes would have the array, byte and
> primitive bits set. The set of recognised data types can of course be
> expanded.
... but that looks rather useful (at least based on the cached version
of that wiki that Google shows ;) - no wiki access right now :( ),
particularly the way you can combine different bytes to create composed
types. I don't see a problem with including this in Hot Rod.
I suppose you included type length into length field so that in the
future you can support other types and possibly longer type fields?
>
>>> - Is the full message size still there as a header field? Is this
>>> necessary? It precludes you from generating messages where you
>>> don't know the full size of the message beforehand. For example,
>>> in the future you might want to write a map of the data in the
>>> cache. I do this, and can chunk the data, thus allowing me to send
>>> an arbitrarily large map. This becomes, to put it mildly, difficult
>>> if you need the size of the request as a header field.
>>
>> +1. If there is a way to work without this size
>> header that would be better.
>
> In my implementation I have a size attached to each field, and this
> allows the messages to be handled easily. I retrieve more data from
> the network if there is not enough data to complete the processing of
> a field. There is no need to know the size of the full message.
I borrowed the idea from the memcached binary protocol, but you have a a
good point. I'll revisit the model to include size together with each field.
Finally, I've noted that you add cache name in the requests. This is
interesting because until now, I had thought of a Hot Rod server to map
1 to 1 to a cache, but if you add the cache name, you can map 1 Hot Rod
server to a Cache manager and the cache allows you to direct requests to
different caches under the same cache manager.
In the case of memcached, since it's a well defined protocol, the
mapping is between 1 to 1 between a Memcached server and a cache.
>
> Thanks,
> Alex
> --- On Mon, 1/4/10, Manik Surtani<manik at jboss.org> wrote:
>
>> From: Manik Surtani<manik at jboss.org>
>> Subject: Re: [infinispan-dev] Hot Rod - pt3
>> To: "infinispan -Dev List"<infinispan-dev at lists.jboss.org>
>> Date: Monday, January 4, 2010, 11:17 AM
>>
>> On 29 Dec 2009, at 18:56, Alex Kluge wrote:
>>
>>> Hi,
>>>
>>> I hope you have all enjoyed the holiday season.
>>>
>>> The protocol is an interesting read. There are a
>> few things that stand
>>> out.
>>>
>>> - No events. I used events, that is asynchronous
>> messages originating
>>> from the server when certain conditions
>> are met, to notify clients
>>> that data was written to the
>> server. This was mostly driven by the
>>> need to notify an L1 cache of a write to
>> the cache server. It would
>>> be good to allow for this usage in this
>> protocol. Note that this is
>>> another case where the op code is useful
>> to have as part of the
>>> message.
>>
>> Isn't this expensive? Doesn't this mean the server
>> has to fire off messages to each and every connected client,
>> whether the clients are interested in these messages or
>> not?
>>
>>> - What happens if the key or the value is not
>> text? I have a way of
>>> representing the data to allow for a wide
>> variety of data types,
>>> even allowing for arrays or maps. This
>> will make the protocol more
>>> complex, but the assumption that the data
>> is a string is rather
>>> limiting. This is already sketched out in
>> the wiki.
>>
>> There is some work around non-text data in the RESTful
>> cache server. Adding MIME types to any opaque data
>> allows us to attach handlers to this data, such as XML or
>> JSON data. What do you have in mind?
>>
>>> - Is the full message size still there as a
>> header field? Is this
>>> necessary? It precludes you from
>> generating messages where you
>>> don't know the full size of the message
>> beforehand. For example,
>>> in the future you might want to write a
>> map of the data in the
>>> cache. I do this, and can chunk the data,
>> thus allowing me to send
>>> an arbitrarily large map. This becomes,
>> to put it mildly, difficult
>>> if you need the size of the request as a
>> header field.
>>
>> +1. If there is a way to work without this size
>> header that would be better.
>>
>> Cheers
>> Manik
>>
>>>
>>>
>>
>> Thanks,
>>>
>>
>> Alex
>>>
>>> --- On Mon, 12/21/09, Manik Surtani<manik at jboss.org>
>> wrote:
>>>
>>>> From: Manik Surtani<manik at jboss.org>
>>>> Subject: Re: [infinispan-dev] Hot Rod - pt3
>>>> To: "infinispan -Dev List"<infinispan-dev at lists.jboss.org>
>>>> Date: Monday, December 21, 2009, 5:36 AM
>>>>
>>>> On 21 Dec 2009, at 11:08, Galder Zamarreno wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Re: http://community.jboss.org/wiki/HotRodProtocol
>>>>>
>>>>> First, I've made the corresponding changes
>> based on
>>>> the feedback I got
>>>>> from pt2. This included reducing the response
>> header
>>>> since clients are
>>>>> already aware of what they sent, addition of
>> topology
>>>> view id to to non
>>>>> dumb requests and further specification in
>> responses
>>>> when topology
>>>>> changes have happened...etc.
>>>>>
>>>>> I've also added flags to the request header
>> that allow
>>>> sending
>>>>> Infinispan flags like: skip cache store, zero
>> lock
>>>> acquisition
>>>>> timeout...etc.
>>>>
>>>>> Note that I've noted this as being N * 1 byte
>> where
>>>> each byte represents
>>>>> a flag. However, I think this could maybe be
>> sent more
>>>> efficiently by
>>>>> using XOR, i.e.
>>>>>
>>>>> 0x00 -> no flag
>>>>> 0x01 (0000 0001) -> zero lock acquisition
>>>>> 0x02 (0000 0010) -> cache mode local
>>>>> 0x03 (0000 0011) -> zero lock acquisition +
>> cache
>>>> mode local
>>>>> 0x04 (0000 0100) -> skip locking
>>>>> ...etc.
>>>>>
>>>>> With 2 bytes, we could implement 16 Flags, we
>>>> currently 11. However, we
>>>>> could use vint as well, making sure that the
>> most
>>>> significant bit does
>>>>> not mean anything flag wise. Iow, with vint,
>> in 1 byte
>
>>>> we'd be able to
>>>>> define 7 diff flags. Thoughts?
>>>>
>>>> +1
>>>>
>>>>> I've also added the quit command that
>> disconnects
>>>> clients.
>>>>
>>>> Does this have any effect on the servers?
>>>>
>>>>> Finally, as far as I'm concerned, the
>> specification is
>>>> complete. I'm
>>>>> leaving the quiet commands out of this initial
>> scope
>>>> (see
>>>>> http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol).
>>>>
>>>>> Remember that quiet commands could be used so
>> that the
>>>> server buffers
>>>>> responses and only when you send a non-quiet
>> command,
>>>> the server replies
>>>>> with all the pending answers.
>>>>>
>>>>> As you can see at the bottom of the wiki, I've
>> added a
>>>> local only put
>>>>> request/response example so that readers get
>> an idea
>>>> of what a full
>>>>> command looks like. I had received some
>> feedback from
>>>> readers saying
>>>>> that it was difficult to understand how it all
>> fit
>>>> together.
>>>>
>>>> The examples look good, however I would not use
>> the
>>>> LOCAL_ONLY flag since that flag really only has
>> meaning in a
>>>> p2p context. In a client-server context,
>> this flag is
>>>> useless, and probably meaningless. I would
>> suggest a
>>>> different flag for the example, e.g.,
>> ZERO_LOCK_TIMEOUT.
>>>>
>>>>> I'll probably add a couple more examples for
>>>> non-so-dumb and clever
>>>>> request/responses but I'll held them until we
>> have a
>>>> final round of
>>>>> feedback and people can indicate whether they
>> want any
>>>> other examples
>>>>> appearing in the wiki.
>>>>>
>>>>> Cheers,
>>>>> --
>>>>> Galder Zamarreño
>>>>> Sr. Software Engineer
>>>>> Infinispan, JBoss Cache
>>>>>
>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>> --
>>>> Manik Surtani
>>>> manik at jboss.org
>>>> Lead, Infinispan
>>>> Lead, JBoss Cache
>>>> http://www.infinispan.org
>>>> http://www.jbosscache.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> --
>> Manik Surtani
>> manik at jboss.org
>> Lead, Infinispan
>> Lead, JBoss Cache
>> http://www.infinispan.org
>> http://www.jbosscache.org
>>
>>
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
More information about the infinispan-dev
mailing list