[infinispan-dev] Infinispan client/server architecture based on gRPC
Adrian Nistor
anistor at redhat.com
Wed May 30 06:31:29 EDT 2018
Fair point. That's why protobuf's Any has a type url inside, exactly for
such flexibility :
https://github.com/google/protobuf/blob/master/src/google/protobuf/any.proto#L150
Well, it's not a mime-type as per infinspan but close enough.
On 05/30/2018 01:22 PM, Gustavo Fernandes wrote:
>
> On Wed, May 30, 2018 at 10:56 AM, Adrian Nistor <anistor at redhat.com
> <mailto:anistor at redhat.com>> wrote:
>
> The oneof and WrappedMessage solve the same problem but in a
> different way.
> Oneof has the nasty effect that in ties the service model to the
> user data model.
>
>
>
> The user data model is only "static" at storage level (guided by
> configuration), and the user data can travel on the wire in any format
> the user wants [1]
>
> [1]
> https://github.com/infinispan/infinispan/blob/master/client/hotrod-client/src/test/java/org/infinispan/client/hotrod/transcoding/DataFormatTest.java#L109
>
> So better not to assume it will be marshalled and unmarshalled in a
> specific way.
>
> Even if it seems like just one more line of code to add when a new
> user type is introduced, it is one line of code in the wrong place
> because you'll have to re-generate the service. IE user run protoc
> again on OUR IDLs. Should a user do that? This coupling between
> the infinispan's service model and the user's data model bothers me.
>
> WrappedMessage is just a wrapper around an array of bytes +
> information regarding what message type or what scalar type is in
> there. Something very similar to a VARIANT [1]. The reason it is
> needed is explained here [2].
>
> You are correct, this is not a gRPC limitation, it is a by-design
> protobuf protocol limitation, that was very thoughtfully
> introduced to reduce wire level bandwitdth for the common case
> where types are static. Unfortunately it leaves generic/dynamic
> types in mid-air. But it is fairly easy to solve, as you can see
> with WrappedMessage. At the time I introduced WrappedMessage we
> were using protobuf 2.
>
> protobuf 3 introduces type Any, which solves the issue in a
> similar way with WrappedMessage. The difference is Any seems to
> have been created to wrap either a plain byte[] or a message type
> that has been marshalled to a byte[]. No support for scalars in
> sight. Can we solve that? Sure, put a WrappedMessage inside that
> byte[] :)))) That is the reason I did not jump immediately at
> using Any and stayed with WrappedMessage.
>
> Can a 150 lines PoC be a proposal for the ISPN object model? No,
> but we need to explore the pain points of gRPC and protobuf that
> are relevant to our usage, and this thing with genericly typed
> services is one of them.
> I think we already have a good solution in sight, before giving up
> and going with byte[] for key and value as it was suggested
> earlier here. I can make a PR to the grpc PoC to show it by the
> end of the week.
>
> Adrian
>
> [1] https://en.wikipedia.org/wiki/Variant_type
> <https://en.wikipedia.org/wiki/Variant_type>
> [2]
> https://developers.google.com/protocol-buffers/docs/techniques#streaming
> <https://developers.google.com/protocol-buffers/docs/techniques#streaming>
>
>
>
> On 05/30/2018 11:34 AM, Vittorio Rigamonti wrote:
>>
>>
>> On Tue, May 29, 2018 at 8:59 PM, Adrian Nistor
>> <anistor at redhat.com <mailto:anistor at redhat.com>> wrote:
>>
>> So you assume the two are separate, Emmanuel. So do I.
>>
>> But in the current PoC the user data model is directly
>> referenced by the service model interface (KeyMsg and
>> ValueMsg are oneofs listing all possible user application
>> types???). I was assuming this hard dependency was there just
>> to make things simple for the scope of the PoC. But let's not
>> make this too simple because it will stop being useful. My
>> expectation is to see a generic yet fully typed 'cache
>> service' interface that does not depend on the key and value
>> types that come from userland, using maybe
>> 'google.protobuf.Any' or our own 'WrappedMessage' type
>> instead. I'm not sure what to believe now because discussing
>> my hopes and assumptions on the gRPC topic on zulip I think I
>> understood the opposite is desired. Vittorio, please comment
>> on this.
>>
>>
>> Yep that was my design choice. Well my first goal was to keep the
>> framework language independent: to reach that I tried to define
>> in grpc/protobuf as much as possible (that's why I didn't use the
>> Any clause). Then I realized that with very little effort I could
>> design a framework that works only with user data from the user
>> side to the cache storage and I'd liked to investigate this,
>> manly for two reasons:
>>
>> - from the user point of view I like the idea that I can found my
>> objects types in the cache
>> - the embeddedCache<object,object> is transparently exposed
>>
>> but this is my 150 lines of code grpc server prototype, not a
>> proposal for the ISPN object model. However it's ok to use it as
>> starting point for a wider discussion
>>
>>
>> I'm still hoping we want to keep the service interface
>> generic and separated from the user model. And if we do it,
>> would you expect to be able to marshall the service call
>> using gRPC lib and at the same time be able to marshall the
>> user model using whatever other library? Would be nice but
>> that seems to be a no-no with gRPC, or I did not search deep
>> enough. I only looked at the java implementation anyway. It
>> seems to be forcing you to go with protoc generated code and
>> protobuf-java.jar all the way, for marshalling both the
>> service and its arguments. And this goes infinitely deeper.
>> If a service argument of type A has a nested field of type B
>> and the marshaller for A is generated with protobuf-java then
>> so is B. Using oneofs or type 'Any' still do not save you
>> from this. The only escape is to pretend the user payload is
>> of type 'bytes'. At that point you are left to do your
>> marshaling to and from bytes yourself. And you are also left
>> with the question, what the heck is the contents of that byte
>> array next time you unmarshall it, which is currently
>> answered by WrappedMessage.
>>
>> And indeed the "oneof" clause in my message definition solves the
>> same problem solved by the WrappedMessage message: what I have to
>> do with these bytes? Actually I'm not sure this is a gRPC
>> limitation: if I receive a stream of bytes I also need some info
>> on what I have to reconstruct.... I'm just guessing
>>
>>
>> So the more I look at gRPC it seems elegant for most purposes
>> but lacking for ours. And again, as with protocol buffers,
>> the wire protocol and the IDL are really nice. It is the
>> implementation that is lacking, IMHO.
>>
>> I think to be really on the same page we should first make a
>> clear statement of what we intend to achieve here in a bit
>> more detail. Also, since this is not a clean slate effort, we
>> should think right from the start what are the expected
>> interactions with existing code base, like what are we
>> willing to sacrifice. Somebody mention hot rod please!
>>
>> Adrian
>>
>>
>>
>> On 05/29/2018 07:20 PM, Emmanuel Bernard wrote:
>>> Right. Here we are talking about a gRPC representation of
>>> the client server interactions. Not the data schema stored
>>> in ISPN. In that model, the API is compiled by us and handed
>>> over as a package.
>>>
>>> On 29 May 2018, at 15:51, Sanne Grinovero
>>> <sanne at infinispan.org <mailto:sanne at infinispan.org>> wrote:
>>>
>>>>
>>>>
>>>> On 29 May 2018 at 13:45, Vittorio Rigamonti
>>>> <vrigamon at redhat.com <mailto:vrigamon at redhat.com>> wrote:
>>>>
>>>> Thanks Adrian,
>>>>
>>>> of course there's a marshalling work under the cover
>>>> and that is reflected into the generated code
>>>> (specially the accessor methods generated from the
>>>> oneof clause).
>>>>
>>>> My opinion is that on the client side this could be
>>>> accepted, as long as the API are well defined and
>>>> documented: application developer can build an adhoc
>>>> decorator on the top if needed. The alternative to this
>>>> is to develop a protostream equivalent for each
>>>> supported language and it doesn't seem really feasible
>>>> to me.
>>>>
>>>>
>>>> This might indeed be reasonable for some developers, some
>>>> languages.
>>>>
>>>> Just please make sure it's not the only option, as many
>>>> other developers will not expect to need a compiler at hand
>>>> in various stages of the application lifecycle.
>>>>
>>>> For example when deploying a JPA model into an appserver,
>>>> or just booting Hibernate in JavaSE as well, there is a
>>>> strong expectation that we'll be able - at runtime - to
>>>> inspect the listed Java POJOs via reflection and
>>>> automatically generate whatever Infinispan will need.
>>>>
>>>> Perhaps a key differentiator is between invoking Infinispan
>>>> APIs (RPC) vs defining the object models and related CODECs
>>>> for keys, values, streams and query results? It might get a
>>>> bit more fuzzy to differentiate them for custom functions
>>>> but I guess we can draw a line somewhere.
>>>>
>>>> Thanks,
>>>> Sanne
>>>>
>>>>
>>>> On the server side (java only) the situation is
>>>> different: protobuf is optimized for streaming not for
>>>> storing so probably a Protostream layer is needed.
>>>>
>>>> On Mon, May 28, 2018 at 4:47 PM, Adrian Nistor
>>>> <anistor at redhat.com <mailto:anistor at redhat.com>> wrote:
>>>>
>>>> Hi Vittorio,
>>>> thanks for exploring gRPC. It seems like a very
>>>> elegant solution for exposing services. I'll have a
>>>> look at your PoC soon.
>>>>
>>>> I feel there are some remarks that need to be made
>>>> regarding gRPC. gRPC is just some nice cheesy
>>>> topping on top of protobuf. Google's implementation
>>>> of protobuf, to be more precise.
>>>> It does not need handwritten marshallers, but the
>>>> 'No need for marshaller' does not accurately
>>>> describe it. Marshallers are needed and are
>>>> generated under the cover by the library and so are
>>>> the data objects and you are unfortunately forced
>>>> to use them. That's both the good news and the bad
>>>> news:) The whole thing looks very promising and
>>>> friendly for many uses cases, especially for demos
>>>> and PoCs :))). Nobody wants to write those
>>>> marshallers. But it starts to become a nuisance if
>>>> you want to use your own data objects.
>>>> There is also the ugliness and excessive memory
>>>> footprint of the generated code, which is the
>>>> reason Infinispan did not adopt the protobuf-java
>>>> library although it did adopt protobuf as an
>>>> encoding format.
>>>> The Protostream library was created as an
>>>> alternative implementation to solve the
>>>> aforementioned problems with the generated code. It
>>>> solves this by letting the user provide their own
>>>> data objects. And for the marshallers it gives you
>>>> two options: a) write the marshaller yourself
>>>> (hated), b) annotated your data objects and the
>>>> marshaller gets generated (loved). Protostream does
>>>> not currently support service definitions right now
>>>> but this is something I started to investigate
>>>> recently after Galder asked me if I think it's
>>>> doable. I think I'll only find out after I do it:)
>>>>
>>>> Adrian
>>>>
>>>>
>>>> On 05/28/2018 04:15 PM, Vittorio Rigamonti wrote:
>>>>> Hi Infinispan developers,
>>>>>
>>>>> I'm working on a solution for developers who need
>>>>> to access Infinispan services through different
>>>>> programming languages.
>>>>>
>>>>> The focus is not on developing a full featured
>>>>> client, but rather discover the value and the
>>>>> limits of this approach.
>>>>>
>>>>> - is it possible to automatically generate useful
>>>>> clients in different languages?
>>>>> - can that clients interoperate on the same cache
>>>>> with the same data types?
>>>>>
>>>>> I came out with a small prototype that I would
>>>>> like to submit to you and on which I would like to
>>>>> gather your impressions.
>>>>>
>>>>> You can found the project here [1]: is a
>>>>> gRPC-based client/server architecture for
>>>>> Infinispan based on and EmbeddedCache, with very
>>>>> few features exposed atm.
>>>>>
>>>>> Currently the project is nothing more than a poc
>>>>> with the following interesting features:
>>>>>
>>>>> - client can be generated in all the grpc
>>>>> supported language: java, go, c++ examples are
>>>>> provided;
>>>>> - the interface is full typed. No need for
>>>>> marshaller and clients build in different language
>>>>> can cooperate on the same cache;
>>>>>
>>>>> The second item is my preferred one beacuse it
>>>>> frees the developer from data marshalling.
>>>>>
>>>>> What do you think about?
>>>>> Sounds interesting?
>>>>> Can you see any flaw?
>>>>>
>>>>> There's also a list of issues for the future [2],
>>>>> basically I would like to investigate these questions:
>>>>> How far this architecture can go?
>>>>> Topology, events, queries... how many of the
>>>>> Infinispan features can be fit in a grpc architecture?
>>>>>
>>>>> Thank you
>>>>> Vittorio
>>>>>
>>>>> [1] https://github.com/rigazilla/ispn-grpc
>>>>> <https://github.com/rigazilla/ispn-grpc>
>>>>> [2] https://github.com/rigazilla/ispn-grpc/issues
>>>>> <https://github.com/rigazilla/ispn-grpc/issues>
>>>>>
>>>>> --
>>>>>
>>>>> Vittorio Rigamonti
>>>>>
>>>>> Senior Software Engineer
>>>>>
>>>>> Red Hat
>>>>>
>>>>> <https://www.redhat.com>
>>>>>
>>>>> Milan, Italy
>>>>>
>>>>> vrigamon at redhat.com <mailto:vrigamon at redhat.com>
>>>>>
>>>>> irc: rigazilla
>>>>>
>>>>> <https://red.ht/sig>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> <mailto:infinispan-dev at lists.jboss.org>
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>> <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Vittorio Rigamonti
>>>>
>>>> Senior Software Engineer
>>>>
>>>> Red Hat
>>>>
>>>> <https://www.redhat.com>
>>>>
>>>> Milan, Italy
>>>>
>>>> vrigamon at redhat.com <mailto:vrigamon at redhat.com>
>>>>
>>>> irc: rigazilla
>>>>
>>>> <https://red.ht/sig>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> <mailto:infinispan-dev at lists.jboss.org>
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>>>>
>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> <mailto:infinispan-dev at lists.jboss.org>
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> <mailto:infinispan-dev at lists.jboss.org>
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>> <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>>
>>
>>
>>
>>
>> --
>>
>> Vittorio Rigamonti
>>
>> Senior Software Engineer
>>
>> Red Hat
>>
>> <https://www.redhat.com>
>>
>> Milan, Italy
>>
>> vrigamon at redhat.com <mailto:vrigamon at redhat.com>
>>
>> irc: rigazilla
>>
>> <https://red.ht/sig>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> <https://lists.jboss.org/mailman/listinfo/infinispan-dev>
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20180530/d034c807/attachment-0001.html
More information about the infinispan-dev
mailing list