[infinispan-dev] protobuf as a marshalling format for infinispan remote-query

Hi all, as some probably know protobuf was chosen as the serialization format for the remote query [1]. Them main reason for choosing it was that is /simple/, time tested, has good support for schema evolution, is nearly ubiquitous (some people say google uses it :)), has multiple language support and most importantly it mandates the existence of a schema for our objects - the proto file. We need that schema on the server side to be able to extract indexable fields from those binary cache values and index them without the need to unmarshall them into plain java domain objects. And I need to stress that the format was chosen, rather than the actual API/library provided by Google [2]. While the protobuf wire format is superb, Google's approach to create a library for marshaling objects to/from a protobuf stream is heavily based on code generation via the protoc tool. Both the marshalling code /and the entities/ to be marshaled (your beloved domain model) are generated. This does not work well if you want to bring your own domain classes to the party. So what we did is create a small set of support classes on top of google's low level wire format classes to assist users in marshaling their own domain model to the protobuf wire format without using google's code generator. These attempts are hosted on a small github project experiment [3] that will be moved to infinispan once we have a conclusion. This project contains several modules demonstrating the attempts. [4] explains the purpose of each module. So far, the approach in module /stream-like/ offers the best user experience. I would name this plan A, which is going to be implemented for infinispan 6.0. Plan A can also use the classes generated by google's protoc code generator tool. So if somebody prefers to go old school it can still work well. Also, the amount of new support code we added on top of google's library is small, so I don't foresee any nightmare in porting this to another language. I would like to get some feedback from anyone who has some time to have a look at [3], specifically at the /stream-like/ module. Stream-like is about 89.9% implemented. There are still some unimplemented methods but do not mind them for the review. And finally, if anyone could suggest a better name for the stream-like package? I can't think of any other option except /streamlike/ :) (which might be trade-marked) So any other options? If no options then we'll just call it /marshaling/. I'll move this to the ispn branch once I know the package name :) Have a nice weekend guys! -------------- [1] https://community.jboss.org/wiki/RemoteQueryDesignInInfinispan [2] https://developers.google.com/protocol-buffers/ [3] https://github.com/anistor/protobuf-playground [4] https://github.com/anistor/protobuf-playground/blob/master/README.md