[infinispan-dev] Hot Rod encoding

Wed Feb 3 08:07:24 EST 2016

Hi all,

Experimenting with custom marshallers is possible right now. People just 
loose query support doing so. Freedom has its cost ... (kidding)

There are a few simple things we can do right now to better support at 
least two client categories (and not loose query support) without much 
changes to current state of the code:
1. provide json support on top of protobuf, via a canonical json <-> 
protobuf translation. We've been talking about this since ispn 7.x. 
Meanwhile protobuf3 is out and provides it out of the box, as Gustavo 
pointed out, so there is even less effort. This would be useful for both 
REST and hot rod.
2. better support for remote query for java clients using 
jbossmarshalling (in compat mode only). Would require hibernate-search 
annotated entities be available on server too, but would spare the user 
from touching protobuf if he's not interested in interop. The only 
reason this does not work already is because it was not among the 
initial set of requirements when designing remote query. 2.5 years later 
it is clearly a must.

Infinispan 9 is not too far, so I propose to implement 1 and 2 as a 
first step, then see what is the right time frame for the more involved 
step of decoupling query module's source of metadata form the protobuf 
schema/marshaller. I think this refactoring deserves to be done even for 
the sake of elegance and maintainability, if not for plugability.

Re Protostream, it is just plain Protobuf folks! It's in no way a 
customly-modified-encoding-loosely-based-on-protobuf. The only apparent 
twist is our usage of Protobuf, which btw also follows Google's 
recommendations (see [1]). All keys and values are wrapped into a struct 
like this [2] (actually it is a union when looking closer). And for good 
reason. Adding that metadata as a separate header at hot rod protocol 
level would be another possibility, but really, having it stored in 
cache with the key/value is much more appropriate. This type metadata 
(the type name or type id) is needed for indexing too. And could also be 
used by some smart cache stores to make some sense out of that blob and 
map it to database columns or suchlike. And would also be needed by non 
HR clients, like REST. I can go on .. :)  The only downside I see is 
space. Our caches are not homogeneous, unfortunately [let's not debate 
now why], so the type info needs to be stored for each single key, value 
instead of storing it once per cache.

Adrian

[1] 
https://developers.google.com/protocol-buffers/docs/techniques#self-description
[2] 
https://raw.githubusercontent.com/anistor/protostream/master/core/src/main/resources/org/infinispan/protostream/message-wrapping.proto

On 02/03/2016 01:34 PM, Sanne Grinovero wrote:
> It sounds like a good idea, almost like a natural evolution, but to
> play devil's advocate I'll try to find some drawbacks for such a
> decision.
>
> One negative argument is overall complexity: there are many points in
> code in which one needs to consider that the encoding might be
> "something else". This isn't a new problem as we already support two
> modes, but we've soon how much more difficult it makes things and this
> is making things even a bit more complex.
>
> Another point which I don't like much is that people will have to
> reconfigure the server depending on specific needs of the clients; if
> we go this way there should be a way in the Hot Rod protocol to
> "negotiate encoding", so to avoid such configuration tasks for end
> users (at least when there's no need to actually deploy dependencies
> on the server to handle some custom encoder..).
>
> The other problem I see is that this severely complicates
> interoperability between different clients. Suppose two applications
> are being developed to use the Infinispan Server and decide to use two
> different encoders, I suspect that at some point they'll regret it
> when needing one to access some data from the other... not suggesting
> that this should be a good practice, but still it would be very
> inconvenient.
>
> Finally, tooling. We'll eventually need to work on better tooling and
> supporting multiple encoders would spread efforts thinly.
>
> That said, I'm not against the idea of toying with such options and
> make other encoders as an *experimental* feature: protobuf is the most
> suited choice but forever is a long time and we should not hinder
> research on better alternatives.
>
> Thanks,
> Sanne
>
>
>
>
> On 3 February 2016 at 10:38, Gustavo Fernandes <gustavo at infinispan.org> wrote:
>>
>> On Mon, Jan 25, 2016 at 2:03 PM, Galder Zamarreño <galder at redhat.com> wrote:
>>> Hi all,
>>>
>>> As I write the Javascript client for Hot Rod, and Vittorio writes the C++
>>> client, the question the encoding of the byte arrays has popped up.
>>>
>>> The reason why encoding matters is mainly because of compatibility mode.
>>> How does a Hot Rod client know how it should transform something a REST
>>> client set?
>>>
>>> To be able to answer this question correctly, the Hot Rod client needs to
>>> know the type of the data.
>>>
>>> Although we could consider adding encoding information to the Hot Rod
>>> protocol long term, in the short term this question might already been
>>> answered by Protostream.
>>
>>
>> Compatibility between disparate clients should certainly be supported, but
>> what about a pluggable marshaller mechanism? Let's consider all usage
>> scenarios:
>>
>> * Only Java clients: in most cases, JBoss marshalling is adequate, no need
>> to involve protobuf or json and since JBoss Marshalling is default, no need
>> to configure anything.
>>
>> * Mix of Java and C++/C#/Python clients: the Protobuf encoding works
>> wonderfully for that. The client could be configured to use the protostream
>> marshaller, the same for the server.
>>
>> * Mix of Java, C++/C#/Python and REST clients: protobuf with json [1]
>> encoding is an interesting option. Same as above, a protostream marshaller
>> with json support could
>> be configured both on client and server.
>>
>> * Custom marshallers: consider FlatBuffers [2], for example, an interesting
>> new cross-platform serializer that allows to access serialized data without
>> de-serializing the whole payload,
>> and without generating any transient memory, and it optionally supports
>> JSON. It could be interesting to write a marshaller based on it and plug it
>> on Infinispan.
>>
>> Although we have a way of configuring the marshaller on client (via
>> RemoteCacheManager) and server (deployable jar), there'd be more work to do
>> in order make them really pluggable:
>>
>> - Some serialization formats like Protobuf and [2] require interface
>> description to be available on both client and server. This is not supported
>> for all marshallers, just for Protostream
>> - Remote query requires some metadata regarding which fields to index and
>> how, and this would need to be possible on every language regardless of the
>> marshalling.
>> Currently, this info can be encoded in the .proto file only, a custom
>> marshaller would not work.
>> - Currently remote query is settled on protobuf not only for the byte array
>> but for the the HotRod query request and response [3]
>>
>> Is having a pluggable serializer mechanism too far fetched, given the effort
>> already invested towards protobuf?
>>
>> Gustavo
>>
>> [1] https://developers.google.com/protocol-buffers/docs/proto3#json
>> [2] https://google.github.io/flatbuffers/
>> [2]
>> https://code.facebook.com/posts/872547912839369/improving-facebook-s-performance-on-android-with-flatbuffers/
>> [3]
>> https://github.com/infinispan/infinispan/blob/master/remote-query/remote-query-client/src/main/resources/org/infinispan/query/remote/client/query.proto
>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev