[infinispan-dev] HotRod: Digging further into the encoding of data

[Manik Surtani]

Not sure I understand.  Why does this need to be as complex as this?  When you say encoding of data, surely all this data is, is a byte[]?

On 18 Feb 2010, at 15:26, Galder Zamarreno wrote:

> Hi,
> 
> I've been looking again at the encoding of data in Hot Rod  
> (http://community.jboss.org/wiki/HotRodProtocol) and there's a few things  
> I'm not too happy about or they're not totally clear:
> 
> 1. The type appears to be pretty wasteful because the majority of types  
> cannot combined with others, for example: You cannot have a type that is  
> Boolean and Long at the same time, or Double and Character. However, you  
> can potentially have an Arrays of Serialized. So, I propose instead  
> separating between a meta type and type.
> 
> Meta types would be: Array, Map, Primitive and Compressed. The meta type  
> would be encoded using bit ops, so you could combine them in diff ways,  
> i.e. Array of primitives. I don't think we should support combinations off  
> diff collections, i.e Map of Array, or Array of Maps. It would complexity  
> and don't forsee an immediate req for this.
> 
> Type would be: Byte, Boolean, Character, String, Date, Double, Float,  
> Integer, Long, Short, Serialized, StringBuilder, and Any. These would  
> literals from 1 to N. Note that I've added Any to separate between two  
> different collections. For example, if you send an Array of String, each  
> individual element just follows together with its size. However, if you  
> send an Array of Any, each individual entry must define its type.
> 
> For Maps, we've got two options: First, no type assumptions made and let  
> each key/value define its own type. Or allow maps meta-type definitions to  
> be followed by not one but two type fields. Even if the map was of mixed  
> types, you could have Any, Any. My preference is for the latter.
> 
> Both type and metatype would be variable length integers.
> 
> 2. Serialized will be stored as byte[] internally, no attempt to  
> unmarshalling will be done in Hot Rod. Clients decide how they wanna  
> marshall this Serialized types. They just need to gives us a byte[] and  
> its length.
> 
> 3. To clarify something that Alex mentioned in the previous encoding data  
> email, Arrays and Maps are followed by the number of items in Hot Rod and  
> not the number of bytes. In case of Arrays of Any, each individual field  
> gives us its size, type and the data. In case of Array of Booleans, each  
> individual field comes with size and data. Size might have been optional  
> in each field of Array of Boolean, but it simplifies deadling with Array  
> of Serialized, where each individual field is a byte[] of arbitrary length.
> 
> For Maps, a similar thing happens. If we have a Map of String, Boolean, we  
> get key/value pair like: k=[size+data]v=[size+data]. If it's a Map of Any,  
> Any, we get k=[type+size+data],v=[size+data]


--
Manik Surtani
manik at jboss.org
Lead, Infinispan
Lead, JBoss Cache
http://www.infinispan.org
http://www.jbosscache.org




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20100218/8cbd2461/attachment-0002.html