<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Wed, May 30, 2018 at 5:00 PM Radim Vansa <<a href="mailto:rvansa@redhat.com">rvansa@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 05/30/2018 02:53 PM, Sanne Grinovero wrote:<br>
> On 30 May 2018 at 13:26, Adrian Nistor <<a href="mailto:anistor@redhat.com" target="_blank">anistor@redhat.com</a>> wrote:<br>
>> Yest, the client needs that hash but that does not necessarily mean it<br>
>> has to compute it itself.<br>
>> The hash should be applied to the storage format which might be<br>
>> different from the format the client sees. So hash computation could be<br>
>> done on the server, just a thought.<br>
> Unless we want to explore some form of hybrid gRPC which benefits from<br>
> Hot Rod intelligence level 3?<br>
<br>
Since Tristan said that gRPC is viable only if the performance is <br>
comparable - I concluded that this involves the smart routing. I was <br>
hoping that gRPC networking layer would provide some hook to specify the <br>
destination. </blockquote><div><br></div><div>It does, via SubchannelPicker implementations. It requires key to be sent as HTTP header down the stack so that the SubchannelPicker can extract it.</div><div><br></div><div>SubchannelPicker impl can then apply hash on it and decide based on available channels.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">An alternative would be a proxy hosted on the same node <br>
that would do the routing. </blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
If we're to replace Hot Rod I was expecting the (generated) gRPC client <br>
to be extensible enough to allow us add client-side features (like near <br>
cache, maybe listeners would need client-side code too) but saving us <br>
most of the hassle with networking and parsing, while providing basic <br>
client in languages we don't embrace without additional cost.<br>
<br>
R.<br>
<br>
><br>
> In which case the client will need to compute the hash before it can<br>
> hint the network layer were to connect to.<br>
><br>
> Thanks,<br>
> Sanne<br>
><br>
>> On 05/30/2018 02:47 PM, Radim Vansa wrote:<br>
>>> On 05/30/2018 12:46 PM, Adrian Nistor wrote:<br>
>>>> Thanks for clarifying this Galder.<br>
>>>> Yes, the network layer is indeed the culprit and the purpose of this<br>
>>>> experiment.<br>
>>>><br>
>>>> What is the approach you envision regarding the IDL? Should we strive<br>
>>>> for a pure IDL definition of the service? That could be an interesting<br>
>>>> approach that would make it possible for a third party to generate<br>
>>>> their own infinispan grpc client in any new language that we do not<br>
>>>> already offer support, just based on the IDL. And maybe using a<br>
>>>> different grpc implementation if they do not find suitable the one<br>
>>>> from google.<br>
>>>><br>
>>>> I was not suggesting we should do type transformation or anything on<br>
>>>> the client side that would require an extra layer of code on top of<br>
>>>> what grpc generates for the client, so maybe a pure IDL based service<br>
>>>> definition would indeed be possible, without extra helpers. No type<br>
>>>> transformation, just type information. Exposing the type info that<br>
>>>> comes from the server would be enough, a lot better than dumbing<br>
>>>> everything down to a byte[].<br>
>>> I may be wrong but key transformation on client is necessary for correct<br>
>>> hash-aware routing, isn't it? We need to get byte array for each key and<br>
>>> apply murmur hash there (IIUC even when we use protobuf as the storage<br>
>>> format, segment is based on the raw protobuf bytes, right?).<br>
>>><br>
>>> Radim<br>
>>><br>
>>>> Adrian<br>
>>>><br>
>>>> On 05/30/2018 12:16 PM, Galder Zamarreno wrote:<br>
>>>>> On Tue, May 29, 2018 at 8:57 PM Adrian Nistor <<a href="mailto:anistor@redhat.com" target="_blank">anistor@redhat.com</a><br>
>>>>> <mailto:<a href="mailto:anistor@redhat.com" target="_blank">anistor@redhat.com</a>>> wrote:<br>
>>>>><br>
>>>>> Vittorio, a few remarks regarding your statement "...The<br>
>>>>> alternative to this is to develop a protostream equivalent for<br>
>>>>> each supported language and it doesn't seem really feasible to me."<br>
>>>>><br>
>>>>> No way! That's a big misunderstanding. We do not need to<br>
>>>>> re-implement the protostream library in C/C++/C# or any new<br>
>>>>> supported language.<br>
>>>>> Protostream is just for Java and it is compatible with Google's<br>
>>>>> protobuf lib we already use in the other clients. We can continue<br>
>>>>> using Google's protobuf lib for these clients, with or without gRPC.<br>
>>>>> Protostream does not handle protobuf services as gRPC does, but<br>
>>>>> we can add support for that with little effort.<br>
>>>>><br>
>>>>> The real problem here is if we want to replace our hot rod<br>
>>>>> invocation protocol with gRPC to save on the effort of<br>
>>>>> implementing and maintaining hot rod in all those clients. I<br>
>>>>> wonder why the obvious question is being avoided in this thread.<br>
>>>>><br>
>>>>><br>
>>>>> ^ It is not being avoided. I stated it quite clearly when I replied<br>
>>>>> but maybe not with enough detail. So, I said:<br>
>>>>><br>
>>>>>> The biggest problem I see in our client/server architecture is the<br>
>>>>> ability to quickly deliver features/APIs across multiple language<br>
>>>>> clients. Both Vittorio and I have seen how long it takes to implement<br>
>>>>> all the different features available in Java client and port them to<br>
>>>>> Node.js, C/C++/C#...etc. This effort lead by Vittorio is trying to<br>
>>>>> improve on that by having some of that work done for us. Granted, not<br>
>>>>> all of it will be done, but it should give us some good foundations<br>
>>>>> on which to build.<br>
>>>>><br>
>>>>> To expand on it a bit further: the reason it takes us longer to get<br>
>>>>> different features in is because each client implements its own<br>
>>>>> network layer, parses the protocol and does type transformations<br>
>>>>> (between byte[] and whatever the client expects).<br>
>>>>><br>
>>>>> IMO, the most costly things there are getting the network layer right<br>
>>>>> (from experience with Node.js, it has taken a while to do so) and<br>
>>>>> parsing work (not only parsing itself, but doing it in a efficient<br>
>>>>> way). Network layer also includes load balancing, failover, cluster<br>
>>>>> failover...etc.<br>
>>>>><br>
>>>>> From past experience, transforming from byte[] to what the client<br>
>>>>> expects has never really been very problematic for me. What's been<br>
>>>>> difficult here is coming up with encoding architecture that Gustavo<br>
>>>>> lead, whose aim was to improve on the initial compatibility mode.<br>
>>>>> But, with that now clear, understood and proven to solve our issues,<br>
>>>>> the rest in this area should be fairly straightforward IMO.<br>
>>>>><br>
>>>>> Type transformation, once done, is a constant. As we add more Hot Rod<br>
>>>>> operations, it's mostly the parsing that starts to become more work.<br>
>>>>> Network can also become more work if instead of RPC commands you<br>
>>>>> start supporting streams based commands.<br>
>>>>><br>
>>>>> gRPC solves the network (FYI: with key as HTTP header and<br>
>>>>> SubchannelPicker you can do hash-aware routing) and parsing for us. I<br>
>>>>> don't see the need for it to solve our type transformations for us.<br>
>>>>> If it does it, great, but does it support our compatibility<br>
>>>>> requirements? (I had already told Vittorio to check Gustavo on this).<br>
>>>>> Type transformation is a lower prio for me, network and parsing are<br>
>>>>> more important.<br>
>>>>><br>
>>>>> Hope this clarifies better my POV.<br>
>>>>><br>
>>>>> Cheers<br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> Adrian<br>
>>>>><br>
>>>>><br>
>>>>> On 05/29/2018 03:45 PM, Vittorio Rigamonti wrote:<br>
>>>>>> Thanks Adrian,<br>
>>>>>><br>
>>>>>> of course there's a marshalling work under the cover and that is<br>
>>>>>> reflected into the generated code (specially the accessor<br>
>>>>>> methods generated from the oneof clause).<br>
>>>>>><br>
>>>>>> My opinion is that on the client side this could be accepted, as<br>
>>>>>> long as the API are well defined and documented: application<br>
>>>>>> developer can build an adhoc decorator on the top if needed. The<br>
>>>>>> alternative to this is to develop a protostream equivalent for<br>
>>>>>> each supported language and it doesn't seem really feasible to me.<br>
>>>>>><br>
>>>>>> On the server side (java only) the situation is different:<br>
>>>>>> protobuf is optimized for streaming not for storing so probably<br>
>>>>>> a Protostream layer is needed.<br>
>>>>>><br>
>>>>>> On Mon, May 28, 2018 at 4:47 PM, Adrian Nistor<br>
>>>>>> <<a href="mailto:anistor@redhat.com" target="_blank">anistor@redhat.com</a> <mailto:<a href="mailto:anistor@redhat.com" target="_blank">anistor@redhat.com</a>>> wrote:<br>
>>>>>><br>
>>>>>> Hi Vittorio,<br>
>>>>>> thanks for exploring gRPC. It seems like a very elegant<br>
>>>>>> solution for exposing services. I'll have a look at your PoC<br>
>>>>>> soon.<br>
>>>>>><br>
>>>>>> I feel there are some remarks that need to be made regarding<br>
>>>>>> gRPC. gRPC is just some nice cheesy topping on top of<br>
>>>>>> protobuf. Google's implementation of protobuf, to be more<br>
>>>>>> precise.<br>
>>>>>> It does not need handwritten marshallers, but the 'No need<br>
>>>>>> for marshaller' does not accurately describe it. Marshallers<br>
>>>>>> are needed and are generated under the cover by the library<br>
>>>>>> and so are the data objects and you are unfortunately forced<br>
>>>>>> to use them. That's both the good news and the bad news:)<br>
>>>>>> The whole thing looks very promising and friendly for many<br>
>>>>>> uses cases, especially for demos and PoCs :))). Nobody wants<br>
>>>>>> to write those marshallers. But it starts to become a<br>
>>>>>> nuisance if you want to use your own data objects.<br>
>>>>>> There is also the ugliness and excessive memory footprint of<br>
>>>>>> the generated code, which is the reason Infinispan did not<br>
>>>>>> adopt the protobuf-java library although it did adopt<br>
>>>>>> protobuf as an encoding format.<br>
>>>>>> The Protostream library was created as an alternative<br>
>>>>>> implementation to solve the aforementioned problems with the<br>
>>>>>> generated code. It solves this by letting the user provide<br>
>>>>>> their own data objects. And for the marshallers it gives you<br>
>>>>>> two options: a) write the marshaller yourself (hated), b)<br>
>>>>>> annotated your data objects and the marshaller gets<br>
>>>>>> generated (loved). Protostream does not currently support<br>
>>>>>> service definitions right now but this is something I<br>
>>>>>> started to investigate recently after Galder asked me if I<br>
>>>>>> think it's doable. I think I'll only find out after I do it:)<br>
>>>>>><br>
>>>>>> Adrian<br>
>>>>>><br>
>>>>>><br>
>>>>>> On 05/28/2018 04:15 PM, Vittorio Rigamonti wrote:<br>
>>>>>>> Hi Infinispan developers,<br>
>>>>>>><br>
>>>>>>> I'm working on a solution for developers who need to access<br>
>>>>>>> Infinispan services through different programming languages.<br>
>>>>>>><br>
>>>>>>> The focus is not on developing a full featured client, but<br>
>>>>>>> rather discover the value and the limits of this approach.<br>
>>>>>>><br>
>>>>>>> - is it possible to automatically generate useful clients<br>
>>>>>>> in different languages?<br>
>>>>>>> - can that clients interoperate on the same cache with the<br>
>>>>>>> same data types?<br>
>>>>>>><br>
>>>>>>> I came out with a small prototype that I would like to<br>
>>>>>>> submit to you and on which I would like to gather your<br>
>>>>>>> impressions.<br>
>>>>>>><br>
>>>>>>> You can found the project here [1]: is a gRPC-based<br>
>>>>>>> client/server architecture for Infinispan based on and<br>
>>>>>>> EmbeddedCache, with very few features exposed atm.<br>
>>>>>>><br>
>>>>>>> Currently the project is nothing more than a poc with the<br>
>>>>>>> following interesting features:<br>
>>>>>>><br>
>>>>>>> - client can be generated in all the grpc supported<br>
>>>>>>> language: java, go, c++ examples are provided;<br>
>>>>>>> - the interface is full typed. No need for marshaller and<br>
>>>>>>> clients build in different language can cooperate on the<br>
>>>>>>> same cache;<br>
>>>>>>><br>
>>>>>>> The second item is my preferred one beacuse it frees the<br>
>>>>>>> developer from data marshalling.<br>
>>>>>>><br>
>>>>>>> What do you think about?<br>
>>>>>>> Sounds interesting?<br>
>>>>>>> Can you see any flaw?<br>
>>>>>>><br>
>>>>>>> There's also a list of issues for the future [2], basically<br>
>>>>>>> I would like to investigate these questions:<br>
>>>>>>> How far this architecture can go?<br>
>>>>>>> Topology, events, queries... how many of the Infinispan<br>
>>>>>>> features can be fit in a grpc architecture?<br>
>>>>>>><br>
>>>>>>> Thank you<br>
>>>>>>> Vittorio<br>
>>>>>>><br>
>>>>>>> [1] <a href="https://github.com/rigazilla/ispn-grpc" rel="noreferrer" target="_blank">https://github.com/rigazilla/ispn-grpc</a><br>
>>>>>>> [2] <a href="https://github.com/rigazilla/ispn-grpc/issues" rel="noreferrer" target="_blank">https://github.com/rigazilla/ispn-grpc/issues</a><br>
>>>>>>><br>
>>>>>>> --<br>
>>>>>>><br>
>>>>>>> Vittorio Rigamonti<br>
>>>>>>><br>
>>>>>>> Senior Software Engineer<br>
>>>>>>><br>
>>>>>>> Red Hat<br>
>>>>>>><br>
>>>>>>> <<a href="https://www.redhat.com" rel="noreferrer" target="_blank">https://www.redhat.com</a>><br>
>>>>>>><br>
>>>>>>> Milan, Italy<br>
>>>>>>><br>
>>>>>>> <a href="mailto:vrigamon@redhat.com" target="_blank">vrigamon@redhat.com</a> <mailto:<a href="mailto:vrigamon@redhat.com" target="_blank">vrigamon@redhat.com</a>><br>
>>>>>>><br>
>>>>>>> irc: rigazilla<br>
>>>>>>><br>
>>>>>>> <<a href="https://red.ht/sig" rel="noreferrer" target="_blank">https://red.ht/sig</a>><br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> _______________________________________________<br>
>>>>>>> infinispan-dev mailing list<br>
>>>>>>> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
>>>>>>> <mailto:<a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a>><br>
>>>>>>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>> --<br>
>>>>>><br>
>>>>>> Vittorio Rigamonti<br>
>>>>>><br>
>>>>>> Senior Software Engineer<br>
>>>>>><br>
>>>>>> Red Hat<br>
>>>>>><br>
>>>>>> <<a href="https://www.redhat.com" rel="noreferrer" target="_blank">https://www.redhat.com</a>><br>
>>>>>><br>
>>>>>> Milan, Italy<br>
>>>>>><br>
>>>>>> <a href="mailto:vrigamon@redhat.com" target="_blank">vrigamon@redhat.com</a> <mailto:<a href="mailto:vrigamon@redhat.com" target="_blank">vrigamon@redhat.com</a>><br>
>>>>>><br>
>>>>>> irc: rigazilla<br>
>>>>>><br>
>>>>>> <<a href="https://red.ht/sig" rel="noreferrer" target="_blank">https://red.ht/sig</a>><br>
>>>>> _______________________________________________<br>
>>>>> infinispan-dev mailing list<br>
>>>>> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
>>>>> <mailto:<a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a>><br>
>>>>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> _______________________________________________<br>
>>>>> infinispan-dev mailing list<br>
>>>>> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
>>>>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
>>>><br>
>>>><br>
>>>> _______________________________________________<br>
>>>> infinispan-dev mailing list<br>
>>>> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
>>>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
>> _______________________________________________<br>
>> infinispan-dev mailing list<br>
>> <a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
<br>
<br>
-- <br>
Radim Vansa <<a href="mailto:rvansa@redhat.com" target="_blank">rvansa@redhat.com</a>><br>
JBoss Performance Team<br>
<br>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
</blockquote></div></div>