Re: [infinispan-dev] Hot Rod encoding

On 3 Feb 2016, at 11:38, Gustavo Fernandes <gustavo(a)infinispan.org&gt; wrote: On Mon, Jan 25, 2016 at 2:03 PM, Galder Zamarreño <galder(a)redhat.com&gt; wrote: Hi all, As I write the Javascript client for Hot Rod, and Vittorio writes the C++ client, the question the encoding of the byte arrays has popped up. The reason why encoding matters is mainly because of compatibility mode. How does a Hot Rod client know how it should transform something a REST client set? To be able to answer this question correctly, the Hot Rod client needs to know the type of the data. Although we could consider adding encoding information to the Hot Rod protocol long term, in the short term this question might already been answered by Protostream. Compatibility between disparate clients should certainly be supported, but what about a pluggable marshaller mechanism? Let's consider all usage scenarios: * Only Java clients: in most cases, JBoss marshalling is adequate, no need to involve protobuf or json and since JBoss Marshalling is default, no need to configure anything. * Mix of Java and C++/C#/Python clients: the Protobuf encoding works wonderfully for that. The client could be configured to use the protostream marshaller, the same for the server. * Mix of Java, C++/C#/Python and REST clients: protobuf with json [1] encoding is an interesting option. Same as above, a protostream marshaller with json support could be configured both on client and server. * Custom marshallers: consider FlatBuffers [2], for example, an interesting new cross-platform serializer that allows to access serialized data without de-serializing the whole payload, and without generating any transient memory, and it optionally supports JSON. It could be interesting to write a marshaller based on it and plug it on Infinispan. Although we have a way of configuring the marshaller on client (via RemoteCacheManager) and server (deployable jar), there'd be more work to do in order make them really pluggable: - Some serialization formats like Protobuf and [2] require interface description to be available on both client and server. This is not supported for all marshallers, just for Protostream - Remote query requires some metadata regarding which fields to index and how, and this would need to be possible on every language regardless of the marshalling. Currently, this info can be encoded in the .proto file only, a custom marshaller would not work. - Currently remote query is settled on protobuf not only for the byte array but for the the HotRod query request and response [3] Is having a pluggable serializer mechanism too far fetched, given the effort already invested towards protobuf? Gustavo [1] https://developers.google.com/protocol-buffers/docs/proto3#json [2] https://google.github.io/flatbuffers/ [2] https://code.facebook.com/posts/872547912839369/improving-facebook-s-perf... [3] https://github.com/infinispan/infinispan/blob/master/remote-query/remote-... _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev