[infinispan-dev] Hot Rod encoding

[Gustavo Fernandes]

On Mon, Jan 25, 2016 at 2:03 PM, Galder Zamarreño <galder at redhat.com> wrote:

> Hi all,
>
> As I write the Javascript client for Hot Rod, and Vittorio writes the C++
> client, the question the encoding of the byte arrays has popped up.
>
> The reason why encoding matters is mainly because of compatibility mode.
> How does a Hot Rod client know how it should transform something a REST
> client set?
>
> To be able to answer this question correctly, the Hot Rod client needs to
> know the type of the data.
>
> Although we could consider adding encoding information to the Hot Rod
> protocol long term, in the short term this question might already been
> answered by Protostream.
>


Compatibility between disparate clients should certainly be supported, but
what about a pluggable marshaller mechanism? Let's consider all usage
scenarios:

* Only Java clients: in most cases, JBoss marshalling is adequate, no need
to involve protobuf or json and since JBoss Marshalling is default, no need
to configure anything.

* Mix of Java and C++/C#/Python clients: the Protobuf encoding works
wonderfully for that. The client could be configured to use the protostream
marshaller, the same for the server.

* Mix of Java, C++/C#/Python and REST clients: protobuf with json [1]
encoding is an interesting option. Same as above, a protostream marshaller
with json support could
be configured both on client and server.

* Custom marshallers: consider FlatBuffers [2], for example, an interesting
new cross-platform serializer that allows to access serialized data without
de-serializing the whole payload,
and without generating any transient memory, and it optionally supports
JSON. It could be interesting to write a marshaller based on it and plug it
on Infinispan.

Although we have a way of configuring the marshaller on client (via
RemoteCacheManager) and server (deployable jar), there'd be more work to do
in order make them really pluggable:

- Some serialization formats like Protobuf and [2] require interface
description to be available on both client and server. This is not
supported for all marshallers, just for Protostream
- Remote query requires some metadata regarding which fields to index and
how, and this would need to be possible on every language regardless of the
marshalling.
Currently, this info can be encoded in the .proto file only, a custom
marshaller would not work.
- Currently remote query is settled on protobuf not only for the byte array
but for the the HotRod query request and response [3]

Is having a pluggable serializer mechanism too far fetched, given the
effort already invested towards protobuf?

Gustavo

[1] https://developers.google.com/protocol-buffers/docs/proto3#json
[2] https://google.github.io/flatbuffers/
[2]
https://code.facebook.com/posts/872547912839369/improving-facebook-s-performance-on-android-with-flatbuffers/
[3]
https://github.com/infinispan/infinispan/blob/master/remote-query/remote-query-client/src/main/resources/org/infinispan/query/remote/client/query.proto


>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20160203/0f8e2afe/attachment.html