Hi all, Re: http://community.jboss.org/wiki/HotRodProtocol First, I've made the corresponding changes based on the feedback I got from pt2. This included reducing the response header since clients are already aware of what they sent, addition of topology view id to to non dumb requests and further specification in responses when topology changes have happened...etc. I've also added flags to the request header that allow sending Infinispan flags like: skip cache store, zero lock acquisition timeout...etc.

Note that I've noted this as being N * 1 byte where each byte represents a flag. However, I think this could maybe be sent more efficiently by using XOR, i.e. 0x00 -> no flag 0x01 (0000 0001) -> zero lock acquisition 0x02 (0000 0010) -> cache mode local 0x03 (0000 0011) -> zero lock acquisition + cache mode local 0x04 (0000 0100) -> skip locking ...etc. With 2 bytes, we could implement 16 Flags, we currently 11. However, we could use vint as well, making sure that the most significant bit does not mean anything flag wise. Iow, with vint, in 1 byte we'd be able to define 7 diff flags. Thoughts?

I've also added the quit command that disconnects clients.

Finally, as far as I'm concerned, the specification is complete. I'm leaving the quiet commands out of this initial scope (see http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol). Remember that quiet commands could be used so that the server buffers responses and only when you send a non-quiet command, the server replies with all the pending answers. As you can see at the bottom of the wiki, I've added a local only put request/response example so that readers get an idea of what a full command looks like. I had received some feedback from readers saying that it was difficult to understand how it all fit together.

I'll probably add a couple more examples for non-so-dumb and clever request/responses but I'll held them until we have a final round of feedback and people can indicate whether they want any other examples appearing in the wiki. Cheers, -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Hi, I hope you have all enjoyed the holiday season. The protocol is an interesting read. There are a few things that stand out. - No events. I used events, that is asynchronous messages originating from the server when certain conditions are met, to notify clients that data was written to the server. This was mostly driven by the need to notify an L1 cache of a write to the cache server. It would be good to allow for this usage in this protocol. Note that this is another case where the op code is useful to have as part of the message.

- What happens if the key or the value is not text? I have a way of representing the data to allow for a wide variety of data types, even allowing for arrays or maps. This will make the protocol more complex, but the assumption that the data is a string is rather limiting. This is already sketched out in the wiki.

- Is the full message size still there as a header field? Is this necessary? It precludes you from generating messages where you don't know the full size of the message beforehand. For example, in the future you might want to write a map of the data in the cache. I do this, and can chunk the data, thus allowing me to send an arbitrarily large map. This becomes, to put it mildly, difficult if you need the size of the request as a header field.

Thanks, Alex --- On Mon, 12/21/09, Manik Surtani <manik(a)jboss.org&gt; wrote: > From: Manik Surtani <manik(a)jboss.org&gt; > Subject: Re: [infinispan-dev] Hot Rod - pt3 > To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org&gt; > Date: Monday, December 21, 2009, 5:36 AM > > On 21 Dec 2009, at 11:08, Galder Zamarreno wrote: > >> Hi all, >> >> Re: http://community.jboss.org/wiki/HotRodProtocol >> >> First, I've made the corresponding changes based on > the feedback I got >> from pt2. This included reducing the response header > since clients are >> already aware of what they sent, addition of topology > view id to to non >> dumb requests and further specification in responses > when topology >> changes have happened...etc. >> >> I've also added flags to the request header that allow > sending >> Infinispan flags like: skip cache store, zero lock > acquisition >> timeout...etc. > >> Note that I've noted this as being N * 1 byte where > each byte represents >> a flag. However, I think this could maybe be sent more > efficiently by >> using XOR, i.e. >> >> 0x00 -> no flag >> 0x01 (0000 0001) -> zero lock acquisition >> 0x02 (0000 0010) -> cache mode local >> 0x03 (0000 0011) -> zero lock acquisition + cache > mode local >> 0x04 (0000 0100) -> skip locking >> ...etc. >> >> With 2 bytes, we could implement 16 Flags, we > currently 11. However, we >> could use vint as well, making sure that the most > significant bit does >> not mean anything flag wise. Iow, with vint, in 1 byte > we'd be able to >> define 7 diff flags. Thoughts? > > +1 > >> I've also added the quit command that disconnects > clients. > > Does this have any effect on the servers? > >> Finally, as far as I'm concerned, the specification is > complete. I'm >> leaving the quiet commands out of this initial scope > (see >> http://code.google.com/p/memcached/wiki/MemcacheBinaryProtocol). > >> Remember that quiet commands could be used so that the > server buffers >> responses and only when you send a non-quiet command, > the server replies >> with all the pending answers. >> >> As you can see at the bottom of the wiki, I've added a > local only put >> request/response example so that readers get an idea > of what a full >> command looks like. I had received some feedback from > readers saying >> that it was difficult to understand how it all fit > together. > > The examples look good, however I would not use the > LOCAL_ONLY flag since that flag really only has meaning in a > p2p context. In a client-server context, this flag is > useless, and probably meaningless. I would suggest a > different flag for the example, e.g., ZERO_LOCK_TIMEOUT. > >> I'll probably add a couple more examples for > non-so-dumb and clever >> request/responses but I'll held them until we have a > final round of >> feedback and people can indicate whether they want any > other examples >> appearing in the wiki. >> >> Cheers, >> -- >> Galder Zamarreño >> Sr. Software Engineer >> Infinispan, JBoss Cache >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

>> - No events. I used events, that is asynchronous messages originating >> from the server when certain conditions are met, to notify clients >> that data was written to the server. This was mostly driven by the >> need to notify an L1 cache of a write to the cache server. It would >> be good to allow for this usage in this protocol. Note that this is >> another case where the op code is useful to have as part of the >> message. > > Isn't this expensive? Doesn't this mean the server > has to fire off messages to each and every connected client, > whether the clients are interested in these messages or > not? Not necessarily. There are a number of options. It wouldn't be difficult to allow the clients to register for the events, and then only send them to the interested clients. The events can be sent asynchronously (a separate thread), thus they won't delay the response to a write. Piratically speaking, they aren't that expensive.

On 01/04/2010 10:44 PM, Alex Kluge wrote: >>> - No events. I used events, that is asynchronous messages originating >>> from the server when certain conditions are met, to notify clients >>> that data was written to the server. This was mostly driven by the >>> need to notify an L1 cache of a write to the cache server. It would >>> be good to allow for this usage in this protocol. Note that this is >>> another case where the op code is useful to have as part of the >>> message. >> >> Isn't this expensive? Doesn't this mean the server >> has to fire off messages to each and every connected client, >> whether the clients are interested in these messages or >> not? > > Not necessarily. There are a number of options. It wouldn't be > difficult to allow the clients to register for the events, and > then only send them to the interested clients. The events can > be sent asynchronously (a separate thread), thus they won't > delay the response to a write. Piratically speaking, they aren't > that expensive. I'm still not convinced by this either. What you're talking about sounds like JMS to me :). Right now, the only situation where I can see this being useful is for sending back cluster formation changes but we found simpler way to deal with it and that doesn't require the added complexity.

> In my implementation I have a size attached to each field, and this > allows the messages to be handled easily. I retrieve more data from > the network if there is not enough data to complete the processing of > a field. There is no need to know the size of the full message. I borrowed the idea from the memcached binary protocol, but you have a a good point. I'll revisit the model to include size together with each field. Finally, I've noted that you add cache name in the requests. This is interesting because until now, I had thought of a Hot Rod server to map 1 to 1 to a cache, but if you add the cache name, you can map 1 Hot Rod server to a Cache manager and the cache allows you to direct requests to different caches under the same cache manager.

> > I can see how they can be abused to form a JMS-like message-passing layer, but then a lot of stuff could be open to abuse. Perhaps the events should just be restricted to notifying when a key has changed (i.e., a bit like Invalidation) but with no values/payload passed on, forcing the client to do a GET if the value was required. Once we have the server->client notification mechanism working IMO we should allow the user to decide which notifications he wants to listen to, and not be restrictive about it. Re:key change notifications, I'm not sure that will work with the current ISP architecture: right now the notifications are local, i.e. one will only be notified by the keys changed in the local cache. So if a used want to be notified when the "account" key changed, that will only happen if he is connected to the server on which the "account" key was hashed. Even more, if he will connect to another server, which contains "account", the notification behavior might be different, which might be confusing. Not a protocol design expert, but is it common for this "push" approach for protocols?

On Jan 8, 2010, at 3:13 PM, Manik Surtani wrote: > > On 8 Jan 2010, at 13:05, Mircea Markus wrote: > >>> >>> I can see how they can be abused to form a JMS-like message-passing layer, but then a lot of stuff could be open to abuse. Perhaps the events should just be restricted to notifying when a key has changed (i.e., a bit like Invalidation) but with no values/payload passed on, forcing the client to do a GET if the value was required. >> Once we have the server->client notification mechanism working IMO we should allow the user to decide which notifications he wants to listen to, and not be restrictive about it. Re:key change notifications, I'm not sure that will work with the current ISP architecture: right now the notifications are local, i.e. one will only be notified by the keys changed in the local cache. So if a used want to be notified when the "account" key changed, that will only happen if he is connected to the server on which the "account" key was hashed. Even more, if he will connect to another server, which contains "account", the notification behavior might be different, which might be confusing. >> Not a protocol design expert, but is it common for this "push" approach for protocols? > > > Well, any form of notification beyond keys would be much too expensive. Although this can be hidden from the user by using a proxy Event object which has a key but lazily loads the value when Event.getValue() is invoked. Not sure I get this. Why would be a view change notification more expensive than the key one? > > Re: the global scope of events, this is important/interesting. > > Client ----> ServerA, ServerB, ServerC ... Server Z > > Assuming the client has a connection to A, and registers interest in keys k1... k3, you are correct that ServerA would only be aware of changes on keys located on ServerA and not globally. Solutions may be that ServerA acts as a proxy for the Client and registers for events on other servers on the clients' behalf? No simple answers here I'm afraid... (this sort of oot) I'm thinking of an more advanced feature here, that would also enable us to support Complex Event Processing [1] and continuos query: build-in notification support for key changes. E.g. cache.registerKeyChangeListener("a*", listenerInstance); this is a pseudo-cod, the idea is that the user will be notified whenever an key starting with "a" (e.g. "a1", "aries") is added/updated/modified, on ANY node of the cluster (v.s. the current node, which is currently supported). Implementing this would mean that the consistent hashing function on all nodes should be aware of all these listeners, and , for each altered key they should notify the corresponding listener about the change (do a push to the node where the listener is installed). This would perfectly fit the following use case: An application that needs to query a huge data set very often (e.g. 100 times a second). Instead of running the query each time, we run it once, cache the result, and register as a listener for possible changes of the queried keys. Whenever the cluster listener is notified, it updates the query result.

[1] http://en.wikipedia.org/wiki/Complex_event_processing [2] http://wiki.tangosol.com/display/COH31UG/Continuous+Query > > Cheers > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Perhaps the events should just be restricted to notifying when a key has changed (i.e., a bit like Invalidation) but with no values/payload passed on, forcing the client to do a GET if the value was required.

From: Manik Surtani <manik(a)jboss.org&gt; Subject: Re: [infinispan-dev] Hot Rod - pt3 To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org&gt; Date: Friday, January 8, 2010, 7:13 AM On 8 Jan 2010, at 13:05, Mircea Markus wrote: >> >> I can see how they can be abused to form a JMS-like message-passing layer, but then a lot of stuff could be open to abuse.  Perhaps the events should just be restricted to notifying when a key has changed (i.e., a bit like Invalidation) but with no values/payload passed on, forcing the client to do a GET if the value was required. > Once we have the server->client notification mechanism working IMO we should allow the user to decide which notifications he wants to listen to, and not be restrictive about it. Re:key change notifications, I'm not sure that will work with the current ISP architecture: right now the notifications are local, i.e. one will only be notified by the keys changed in the local cache. So if a used want to be notified when the "account" key changed, that will only happen if he is connected to the server on which the "account" key was hashed.  Even more, if he will connect to another server, which contains "account", the notification behavior might be different, which might be confusing.  > Not a protocol design expert, but is it common for this "push" approach for protocols? Well, any form of notification beyond keys would be much too expensive.  Although this can be hidden from the user by using a proxy Event object which has a key but lazily loads the value when Event.getValue() is invoked. Re: the global scope of events, this is important/interesting. Client ----> ServerA, ServerB, ServerC ... Server Z Assuming the client has a connection to A, and registers interest in keys k1... k3, you are correct that ServerA would only be aware of changes on keys located on ServerA and not globally.  Solutions may be that ServerA acts as a proxy for the Client and registers for events on other servers on the clients' behalf?  No simple answers here I'm afraid... Cheers -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 01/08/2010 08:26 PM, Alex Kluge wrote: >> Perhaps the events should just be restricted to notifying when a key has changed (i.e., a bit like Invalidation) but with no values/payload passed on, forcing the client to do a GET if the value was required. > > Exactly, we can define the set of events that they can listen for, the end > user only has the choice to subscribe to the events or not. I am most interested in being able to listen for write events. The level of granularity though could use some thought. > > > The granularity of the events has also been a topic of discussion. The end user is likely to want to make the selection at the level of the specific cache, or, perhaps, of keys within a cache. However, I expect that they would be well served with making the selection at the level of the cache, rather than trying to sort out which specific keys they are interested in. > > > Relevant events should be seen no mater which server you are connected to. I am trying to stay away from implementation details, but I can say that I have done this, and that the cost is not that high. Just to make sure we're all in the same page. I don't expect to specify key change related events for the moment. Right now, the primary use case that I see is sending back cluster formation changes, rather than passing this back as part of response. This would allow for clients to subscribe to them depending on their capabilities. However, events will be specified in such way that allow for easy extension for other use cases mentioned. Do we all agree with this?

Hi Alex, Wrt the discussion below about encoding data, I wanted to know what exactly COMPRESSED type meant in your wiki, or what exactly it represents. Also, I wanted to know what bits you send when the object sent accross is an instance of java.lang.Long. Apart from marking LONG, do you also marked as SERIALIZED? Or do you just use SERIALIZED for user-specific classes? Cheers, On 01/05/2010 10:52 AM, Galder Zamarreno wrote: > > > On 01/04/2010 10:44 PM, Alex Kluge wrote: >>>> </snip> > >> >>>> - What happens if the key or the value is not text? I have a way of >>>> representing the data to allow for a wide variety of data types, >>>> even allowing for arrays or maps. This will make the protocol more >>>> complex, but the assumption that the data is a string is rather >>>> limiting. This is already sketched out in the wiki. > > Hmmmmm, I don't think I've made any assumptions in the wiki that keys or > values are Strings unless I've made a mistake somewhere (maybe in the > example where I've used a particular encoding for Strings?). My thoughts > around this was that I was gonna treat them both as byte[]... > >>> >>> </snip> >> >> The idea is to prefix each data block with a data type. This is a >> lightweight binary protocol, so a full fledged mime type would be >> overkill. There is a discussion and example in the Encoding Data >> section of this page: >> >> http://community.jboss.org/wiki/RemoteCacheInteractions >> >> Data types are limited to things like integer, byte, string, boolean, etc. >> Or, if it isn't a recognised type, the native platform serialisation is >> used. There can be arrays of these types, or maps as well. >> >> Each data type is represented by a bit, and they can be used in >> combinations. An array of bytes would have the array, byte and >> primitive bits set. The set of recognised data types can of course be >> expanded. > > ... but that looks rather useful (at least based on the cached version > of that wiki that Google shows ;) - no wiki access right now :( ), > particularly the way you can combine different bytes to create composed > types. I don't see a problem with including this in Hot Rod. > > I suppose you included type length into length field so that in the > future you can support other types and possibly longer type fields? >

Wrt the discussion below about encoding data, I wanted to know what exactly COMPRESSED type meant in your wiki, or what exactly it represents.

Also, I wanted to know what bits you send when the object sent accross is an instance of java.lang.Long. Apart from marking LONG, do you also marked as SERIALIZED? Or do you just use SERIALIZED for user-specific classes?

With regards to individual fields in Map/Array, according to your wiki, each field then only contains the size of the field and the field. So, in a map of booleans, each field would be represented as: 0x01 and 0x01/0x00 with 1st being the size and 2nd being the actual value. Correct?

From: Galder Zamarreno <galder(a)redhat.com&gt; Subject: Re: [infinispan-dev] Enconding of data WAS Re: Hot Rod - pt3 To: infinispan-dev(a)lists.jboss.org Date: Tuesday, January 12, 2010, 10:48 AM On 01/12/2010 05:46 PM, Galder Zamarreno wrote: > And another question: > > With regards to individual fields in Map/Array, according to your wiki, > each field then only contains the size of the field and the field. So, > in a map of booleans, each field would be represented as: 0x01 and > 0x01/0x00 with 1st being the size and 2nd being the actual value. Correct? Well, this would be more like an array actually. By the way, in a Map, how do you send key/value pairs? I suppose [key][value][key][value]? And how do you provide type of key vs type of value? > > On 01/12/2010 04:46 PM, Galder Zamarreno wrote: >> Hi Alex, >> >> Wrt the discussion below about encoding data, I wanted to know what >> exactly COMPRESSED type meant in your wiki, or what exactly it represents. >> >> Also, I wanted to know what bits you send when the object sent accross >> is an instance of java.lang.Long. Apart from marking LONG, do you also >> marked as SERIALIZED? Or do you just use SERIALIZED for user-specific >> classes? >> >> Cheers, >> >> On 01/05/2010 10:52 AM, Galder Zamarreno wrote: >>> >>> >>> On 01/04/2010 10:44 PM, Alex Kluge wrote: >>>>>>      </snip> >>> >>>> >>>>>>      - What happens if the key or the value is not text? I have a way of >>>>>>        representing the data to allow for a wide variety of data types, >>>>>>        even allowing for arrays or maps. This will make the protocol more >>>>>>        complex, but the assumption that the data is a string is rather >>>>>>        limiting. This is already sketched out in the wiki. >>> >>> Hmmmmm, I don't think I've made any assumptions in the wiki that keys or >>> values are Strings unless I've made a mistake somewhere (maybe in the >>> example where I've used a particular encoding for Strings?). My thoughts >>> around this was that I was gonna treat them both as byte[]... >>> >>>>> >>>>> </snip> >>>> >>>>      The idea is to prefix each data block with a data type. This is a >>>>      lightweight binary protocol, so a full fledged mime type would be >>>>      overkill. There is a discussion and example in the Encoding Data >>>>      section of this page: >>>> >>>>        http://community.jboss.org/wiki/RemoteCacheInteractions >>>> >>>>      Data types are limited to things like integer, byte, string, boolean, etc. >>>>      Or, if it isn't a recognised type, the native platform serialisation is >>>>      used. There can be arrays of these types, or maps as well. >>>> >>>>      Each data type is represented by a bit, and they can be used in >>>>      combinations. An array of bytes would have the array, byte and >>>>      primitive bits set. The set of recognised data types can of course be >>>>      expanded. >>> >>> ... but that looks rather useful (at least based on the cached version >>> of that wiki that Google shows ;) - no wiki access right now :( ), >>> particularly the way you can combine different bytes to create composed >>> types. I don't see a problem with including this in Hot Rod. >>> >>> I suppose you included type length into length field so that in the >>> future you can support other types and possibly longer type fields? >>> > -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

From: Galder Zamarreno <galder(a)redhat.com&gt; Subject: Re: [infinispan-dev] Enconding of data WAS Re: Hot Rod - pt3 To: "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org&gt; Date: Thursday, February 18, 2010, 6:12 AM Alex, Thanks for the clarifications. One last question, is the implementation  you did open source or closed source? Cheers, On Mon, 15 Feb 2010 21:01:38 +0100, Alex Kluge <java_kluge(a)yahoo.com&gt; wrote: > Hi, > >   There were a few questions, and it will be easier to understand the  > answers with a >  little background on the implementation.  The key and value objects are  > mapped to >  bytes for transmission over the network using an encoder object, an  > instance of >      org.jboss.cache.tcpcache.messages.encoder.BaseEncoder, >  and then decoded by a decoder object, which is an instance of >      org.jboss.cache.tcpcache.messages.decoder.BaseDecoder. >  One of the goals of these objects is to, where possible, map the data  > into a >  representation that can be interpreted on any platform. As long as  > there is no loss of >  information, these classes can be adjusted as needed to encode and  > decode data fields >  efficiently. > >   > Wrt the discussion below about encoding data, I wanted to know what >   > exactly COMPRESSED type meant in your wiki, or what exactly it  > represents. > >  If a value encodes to a byte array longer than a threshold value,  > compressionThreshold >  in the encoder, an instance of java.util.zip.Deflater is used to  > compress the bytes before >  transmission over the wire, and the COMPRESSED flag is set. On receipt,  > the decoder >  sees the COMPRESSED flag and decompresses the bytes, then interprets  > them in the >  normal way. So, compression happens after converting the data to bytes,  > and >  decompression happens before converting from bytes. > >  > Also, I wanted to know what bits you send when the object sent accross >  > is an instance of java.lang.Long. Apart from marking LONG, do you also >  > marked as SERIALIZED? Or do you just use SERIALIZED for user-specific >  > classes? > >  A  java.lang.Long would only have the long bit set. A long (not an  > object) would have >  the long and primitive bits set. An object that the encoder does not  > understand would >  then be serialized using Java object serialization, and the serialized  > bit would be set. >  The long and Long values will be encoded identically, however, the  > decoder will return >  the appropriate type as determined by the flags. > >  > With regards to individual fields in Map/Array, according to your  > wiki, >  > each field then only contains the size of the field and the field. So, >  > in a map of booleans, each field would be represented as: 0x01 and >  > 0x01/0x00 with 1st being the size and 2nd being the actual value.  > Correct? > >  Arrays and maps are treated a bit differently. Arrays are generally of  > a single known >  type, where maps tend to be between arbitrary types of objects. The  > encoding can >  encode the single type of an array, but not the multiple types of a map. > >  As a result, arrays are encoded with a single size field, followed by  > all of the data bytes. >  The current, suboptimal, implementation places a byte count for all of  > the data followed >  by the type flags, followed by the data. Thus an array of two Integers  > would be encoded >  as > >   A byte count of 12, >    four bytes for flags (INTEGER & ARRAY), >    four bytes for integer 1, >    four bytes for integer 2. > >  This requires the whole array of encoded integers be loaded at once.  > Which should not >  be necessary. However, I was a bit pressed for time. > >  A map is of necessity treated differently. The map also starts with the  > total byte >  count, and is then followed by the MAP flag. Each key and value in the  > map are then >  encoded as individual objects with their own, size, type and data  > fields. > >  I would like to find a way of writing only the number of fields at the  > beginning of these, >  however, time pressure and the need to properly handle compressed data  > forced me to >  place the total number of bytes as the first field. >                                                                  Alex > > --- On Tue, 1/12/10, Galder Zamarreno <galder(a)redhat.com&gt; wrote: > >> From: Galder Zamarreno <galder(a)redhat.com&gt; >> Subject: Re: [infinispan-dev] Enconding of data WAS Re:  Hot Rod - pt3 >> To: infinispan-dev(a)lists.jboss.org >> Date: Tuesday, January 12, 2010, 10:48 AM >> >> >> On 01/12/2010 05:46 PM, Galder Zamarreno wrote: >> > And another question: >> > >> > With regards to individual fields in Map/Array, >> according to your wiki, >> > each field then only contains the size of the field >> and the field. So, >> > in a map of booleans, each field would be represented >> as: 0x01 and >> > 0x01/0x00 with 1st being the size and 2nd being the >> actual value. Correct? >> >> Well, this would be more like an array actually. >> >> By the way, in a Map, how do you send key/value pairs? I >> suppose >> [key][value][key][value]? And how do you provide type of >> key vs type of >> value? >> >> > >> > On 01/12/2010 04:46 PM, Galder Zamarreno wrote: >> >> Hi Alex, >> >> >> >> Wrt the discussion below about encoding data, I >> wanted to know what >> >> exactly COMPRESSED type meant in your wiki, or >> what exactly it represents. >> >> >> >> Also, I wanted to know what bits you send when the >> object sent accross >> >> is an instance of java.lang.Long. Apart from >> marking LONG, do you also >> >> marked as SERIALIZED? Or do you just use >> SERIALIZED for user-specific >> >> classes? >> >> >> >> Cheers, >> >> >> >> On 01/05/2010 10:52 AM, Galder Zamarreno wrote: >> >>> >> >>> >> >>> On 01/04/2010 10:44 PM, Alex Kluge wrote: >> >>>>>>      </snip> >> >>> >> >>>> >> >>>>>>      - What happens >> if the key or the value is not text? I have a way of >> >>>>>>      representing the data to allow for a wide variety of data >> types, >> >>>>>>        even >> allowing for arrays or maps. This will make the protocol >> more >> >>>>>>      complex, but the assumption that the data is a string is >> rather >> >>>>>>      limiting. This is already sketched out in the wiki. >> >>> >> >>> Hmmmmm, I don't think I've made any >> assumptions in the wiki that keys or >> >>> values are Strings unless I've made a mistake >> somewhere (maybe in the >> >>> example where I've used a particular encoding >> for Strings?). My thoughts >> >>> around this was that I was gonna treat them >> both as byte[]... >> >>> >> >>>>> >> >>>>> </snip> >> >>>> >> >>>>      The idea is to prefix >> each data block with a data type. This is a >> >>>>      lightweight binary >> protocol, so a full fledged mime type would be >> >>>>      overkill. There is a >> discussion and example in the Encoding Data >> >>>>      section of this page: >> >>>> >> >>>>        http://community.jboss.org/wiki/RemoteCacheInteractions >> >>>> >> >>>>      Data types are limited >> to things like integer, byte, string, boolean, etc. >> >>>>      Or, if it isn't a >> recognised type, the native platform serialisation is >> >>>>      used. There can be >> arrays of these types, or maps as well. >> >>>> >> >>>>      Each data type is >> represented by a bit, and they can be used in >> >>>>      combinations. An array >> of bytes would have the array, byte and >> >>>>      primitive bits set. >> The set of recognised data types can of course be >> >>>>      expanded. >> >>> >> >>> ... but that looks rather useful (at least >> based on the cached version >> >>> of that wiki that Google shows ;) - no wiki >> access right now :( ), >> >>> particularly the way you can combine different >> bytes to create composed >> >>> types. I don't see a problem with including >> this in Hot Rod. >> >>> >> >>> I suppose you included type length into length >> field so that in the >> >>> future you can support other types and >> possibly longer type fields? >> >>> >> > >> >> -- >> Galder Zamarreño >> Sr. Software Engineer >> Infinispan, JBoss Cache >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev