[infinispan-dev] Findings of ISPN-508 - Portable serialization marshaller

[Galder Zamarreño]

The java version of the marshaller is in trunk now. I've added a wiki on this:

http://community.jboss.org/docs/DOC-15774

Once more Hot Rod clients have implemented, we'll look into providing a marshaller for the particular target language.

On Jul 23, 2010, at 6:03 PM, Manik Surtani wrote:

> 
> On 23 Jul 2010, at 15:27, Galder Zamarreño wrote:
> 
>> 
>> On Jul 23, 2010, at 1:47 PM, Manik Surtani wrote:
>> 
>>> Yeah definitely a good idea to break the problem down into specific cases.
>>> 
>>> On 22 Jul 2010, at 10:47, Galder Zamarreño wrote:
>>> 
>>>> Ok, dealing with simple types/collections is clear. We delegate to the corresponding library and that's it.
>>>> 
>>>> As far as custom types that are controlled by user, so I've gone and compared 3 different methods to deal with them:
>>>> 
>>>> Option 1. User/client code generates the byte[] from a precompiled class. Code example: http://pastebin.com/iqX9rVqV
>>>> Option 2. User/client gives us the Pojo and our marshaller converts it into a byte[] with the help of the precompiled class methods. Code example: http://pastebin.com/FTTjcfR4
>>>> Option 3. User/client gives us the Pojo and our marshaller converts it into a byte using client provided serializer/deserializer impl that relies on portable simple type marshaller. I don't have a code example but it'd be very similar to option 2 with the following changes: At startup, ids and class names would be mapped from a given configuration. At writing time, the id would be written and the rest would be delegated to user provided impl. At reading time, once the id has been determined, the class would be instantiated and it would delegate to it the rest of the reading part.
>>>> 
>>>> With this in mind, I've created a spreadsheet comparing them based on different factors: http://spreadsheets.google.com/pub?key=0Ag5RGdzR_GsldEJkUEExX2JPTHdsSF93MkdENkZOUmc&hl=en&output=html
>>>> 
>>>> I'm leaning towards 1 for its simplicity (less code and less configuration in spite of required precompilation) and performance (no extra bytes, no reflection or class loading).
>>> 
>>> I would lean towards 1 as well.
>>> 
>>> Just to re-cap on the cases here:
>>> 
>>> 1) JVM <--> JVM interaction: just use our default marshallers.  Indexing supported.
>> 
>> Yup. At the moment, the default marshaller for HotRod client/server is based on std Java serialization which deals with class indexing.
> 
> Nope, the default (as of CR2) is based on JBoss Marshalling.
> 
> http://fisheye.jboss.org/browse/Infinispan/trunk/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/impl/ConfigurationProperties.java?r=2040#l80
> 
>> We could also plug JBoss Marshalling based marshaller but we'd need the ability for Hot Rod clients to be configurable with user provided externalizers.
> 
> Yup, this is possible.  See above.
> 
>> As a side note, https://jira.jboss.org/browse/ISPN-244 would not be needed here cos Hot Rod server treats stuff as merely a byte[], so no need for deserialization on the Hot Rod server side. But, any client externalizer would be encouraged to follow the pattern in that JIRA.
>> 
>> However, one interesting thing would be how to configure Hot Rod clients that are part of RemoteCacheStore. In this case, you could use https://jira.jboss.org/browse/ISPN-244 as a way to configure both inter Infinispan JVM externalizer marshalling and Hot Rod client. In fact, if you used lazy deserialization here, Hot Rod client would not need any configuration, cos it could take the byte[] in memory and directly ship it to the remote Hot Rod server. 
>> 
>>> 2) JVM <--> non-JVM interaction, just storing primitives + collections.  We provide a marshaller for this?  Indexing supported.
>> 
>> Yeah, we'd need a marshaller to take basic types and common collections that supports indexing. Find attached a very rough version of what it might look like if we used Protobufs.
>> 
>> <BasicTypeMarshallingProtobuf.java>
>> 
>> I'd need to verify whether whatever that marshaller writes can be read successfully read say from an python marshaller equivalent.
>> 
>>> 3) JVM <--> non-JVM interaction, storing custom objects.  User needs to serialize/de-serialize, and store byte[]'s.  Indexing NOT supported.
>> 
>> Yup, no indexing here. Client code knows what it retrieves from the cache and applies the corresponding method, i.e. static method Pojo.parseFrom(). 
>> 
>>> 
>>> Does that sum it up?
>> 
>> Yeah, pretty much.
>> 
>> Something else to decide is which of the libraries out there to go for. I've done a little comparison in: http://spreadsheets.google.com/pub?key=0Ag5RGdzR_GsldDdOVnpwbDUyT20xZlZPZUk5aFNjcGc&hl=en&output=html
>> 
>> I'm still undecided on which one I prefer, particularly cos I just discovered a hidden feature in MessagePack that might make the basic type/collection marshaller very simple to code.
>> 
>> I'll report as soon as I know more.
> 
> Ok, cool!
> 
> Cheers
> Manik
> 
> --
> Manik Surtani
> manik at jboss.org
> Lead, Infinispan
> Lead, JBoss Cache
> http://www.infinispan.org
> http://www.jbosscache.org
> 
> 
> 
> 
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache