<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Tue, May 29, 2018 at 8:57 PM Adrian Nistor <<a href="mailto:anistor@redhat.com" target="_blank">anistor@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div class="m_5548383007427065293m_-557338394167953243moz-cite-prefix">Vittorio, a few remarks regarding your
statement "...The alternative to this is to develop a protostream
equivalent for each supported language and it doesn't seem really
feasible to me."<br>
<br>
No way! That's a big misunderstanding. We do not need to
re-implement the protostream library in C/C++/C# or any new
supported language.<br>
Protostream is just for Java and it is compatible with Google's
protobuf lib we already use in the other clients. We can continue
using Google's protobuf lib for these clients, with or without
gRPC.<br>
Protostream does not handle protobuf services as gRPC does, but we
can add support for that with little effort.<br>
<br>
The real problem here is if we want to replace our hot rod
invocation protocol with gRPC to save on the effort of
implementing and maintaining hot rod in all those clients. I
wonder why the obvious question is being avoided in this thread.</div></div></blockquote><div><br></div></div><div dir="ltr"><div class="gmail_quote"><div>^ It is not being avoided. I stated it quite clearly when I replied but maybe not with enough detail. So, I said:</div></div></div><div dir="ltr"><div class="gmail_quote"><div><br></div><div>> The biggest problem I see in our client/server architecture is the ability to quickly deliver features/APIs across multiple language clients. Both Vittorio and I have seen how long it takes to implement all the different features available in Java client and port them to Node.js, C/C++/C#...etc. This effort lead by Vittorio is trying to improve on that by having some of that work done for us. Granted, not all of it will be done, but it should give us some good foundations on which to build.</div><div><br></div><div>To expand on it a bit further: the reason it takes us longer to get different features in is because each client implements its own network layer, parses the protocol and does type transformations (between byte[] and whatever the client expects). </div><div><br></div><div>IMO, the most costly things there are getting the network layer right (from experience with Node.js, it has taken a while to do so) and parsing work (not only parsing itself, but doing it in a efficient way). Network layer also includes load balancing, failover, cluster failover...etc.</div><div><br></div><div>From past experience, transforming from byte[] to what the client expects has never really been very problematic for me. What's been difficult here is coming up with encoding architecture that Gustavo lead, whose aim was to improve on the initial compatibility mode. But, with that now clear, understood and proven to solve our issues, the rest in this area should be fairly straightforward IMO.</div><div><br></div><div>Type transformation, once done, is a constant. As we add more Hot Rod operations, it's mostly the parsing that starts to become more work. Network can also become more work if instead of RPC commands you start supporting streams based commands.</div><div><br></div><div>gRPC solves the network (FYI: with key as HTTP header and SubchannelPicker you can do hash-aware routing) and parsing for us. I don't see the need for it to solve our type transformations for us. If it does it, great, but does it support our compatibility requirements? (I had already told Vittorio to check Gustavo on this). Type transformation is a lower prio for me, network and parsing are more important.</div><div><br></div><div>Hope this clarifies better my POV.</div><div><br></div><div>Cheers</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><div class="m_5548383007427065293m_-557338394167953243moz-cite-prefix"><br>
<br>
Adrian</div></div><div text="#000000" bgcolor="#FFFFFF"><div class="m_5548383007427065293m_-557338394167953243moz-cite-prefix"><br>
<br>
On 05/29/2018 03:45 PM, Vittorio Rigamonti wrote:<br>
</div></div><div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">
<div>
<div>Thanks Adrian,<br>
<br>
</div>
of course there's a marshalling work under the cover and that
is reflected into the generated code (specially the accessor
methods generated from the oneof clause).<br>
<br>
</div>
<div>My opinion is that on the client side this could be
accepted, as long as the API are well defined and documented:
application developer can build an adhoc decorator on the top
if needed. The alternative to this is to develop a protostream
equivalent for each supported language and it doesn't seem
really feasible to me.<br>
<br>
</div>
<div>On the server side (java only) the situation is different:
protobuf is optimized for streaming not for storing so
probably a Protostream layer is needed.<br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, May 28, 2018 at 4:47 PM, Adrian
Nistor <span dir="ltr"><<a href="mailto:anistor@redhat.com" target="_blank">anistor@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div class="m_5548383007427065293m_-557338394167953243m_4400301142433856756moz-cite-prefix">Hi
Vittorio,<br>
thanks for exploring gRPC. It seems like a very elegant
solution for exposing services. I'll have a look at your
PoC soon.<br>
<br>
I feel there are some remarks that need to be made
regarding gRPC. gRPC is just some nice cheesy topping on
top of protobuf. Google's implementation of protobuf, to
be more precise. <br>
It does not need handwritten marshallers, but the 'No
need for marshaller' does not accurately describe it.
Marshallers are needed and are generated under the cover
by the library and so are the data objects and you are
unfortunately forced to use them. That's both the good
news and the bad news:) The whole thing looks very
promising and friendly for many uses cases, especially
for demos and PoCs :))). Nobody wants to write those
marshallers. But it starts to become a nuisance if you
want to use your own data objects.<br>
There is also the ugliness and excessive memory
footprint of the generated code, which is the reason
Infinispan did not adopt the protobuf-java library
although it did adopt protobuf as an encoding format.<br>
The Protostream library was created as an alternative
implementation to solve the aforementioned problems with
the generated code. It solves this by letting the user
provide their own data objects. And for the marshallers
it gives you two options: a) write the marshaller
yourself (hated), b) annotated your data objects and the
marshaller gets generated (loved). Protostream does not
currently support service definitions right now but this
is something I started to investigate recently after
Galder asked me if I think it's doable. I think I'll
only find out after I do it:)<br>
<br>
Adrian
<div>
<div class="m_5548383007427065293m_-557338394167953243h5"><br>
<br>
On 05/28/2018 04:15 PM, Vittorio Rigamonti wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div class="m_5548383007427065293m_-557338394167953243h5">
<div dir="ltr">
<div>Hi Infinispan developers,<br>
<br>
</div>
<div>I'm working on a solution for developers who
need to access Infinispan services through
different programming languages.<br>
</div>
<div><br>
The focus is not on developing a full featured
client, but rather discover the value and the
limits of this approach.<br>
<div><br>
</div>
</div>
<div>- is it possible to automatically generate
useful clients in different languages?<br>
</div>
<div>- can that clients interoperate on the same
cache with the same data types?<br>
</div>
<div><br>
</div>
<span id="m_5548383007427065293m_-557338394167953243m_4400301142433856756m_-8148977098275501500gmail-result_box" class="m_5548383007427065293m_-557338394167953243m_4400301142433856756m_-8148977098275501500gmail-" lang="en"><span class="m_5548383007427065293m_-557338394167953243m_4400301142433856756m_-8148977098275501500gmail-">I came out
with a small prototype that I would like to
submit to you and on which I would like to
gather your impressions.</span></span><br>
<div><br>
You can found the project here [1]: is a
gRPC-based client/server architecture for
Infinispan based on and EmbeddedCache, with very
few features exposed atm.<br>
</div>
<div><br>
Currently the project is nothing more than a poc
with the following interesting features:<br>
<br>
</div>
<div>- client can be generated in all the grpc
supported language: java, go, c++ examples are
provided;<br>
</div>
<div>- the interface is full typed. No need for
marshaller and clients build in different
language can cooperate on the same cache;<br>
</div>
<div><br>
</div>
<div>The second item is my preferred one beacuse
it frees the developer from data marshalling.<br>
<br>
</div>
<div>What do you think about?<br>
</div>
<div>Sounds interesting?<br>
</div>
<div>Can you see any flaw?<br>
</div>
<div><br>
There's also a list of issues for the future
[2], basically I would like to investigate these
questions:<br>
How far this architecture can go?<br>
Topology, events, queries... how many of the
Infinispan features can be fit in a grpc
architecture?<br>
</div>
<div><br>
</div>
<div>Thank you<br>
</div>
<div>Vittorio<br>
</div>
<br>
[1] <a href="https://github.com/rigazilla/ispn-grpc" target="_blank">https://github.com/rigazilla/ispn-grpc</a><br clear="all">
<div>
<div>
<div>[2] <a href="https://github.com/rigazilla/ispn-grpc/issues" target="_blank">https://github.com/rigazilla/ispn-grpc/issues</a><br clear="all">
<br>
</div>
<div>-- <br>
<div class="m_5548383007427065293m_-557338394167953243m_4400301142433856756m_-8148977098275501500gmail-m_-8431198291286624134m_1982721755148179188gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<p style="font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><span>Vittorio</span>
<span>Rigamonti</span></p>
<p style="font-weight:normal;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><span>Senior
Software Engineer</span></p>
<p style="font-weight:normal;margin:0px;font-size:10px;color:rgb(153,153,153)"><a style="color:rgb(0,136,206);font-size:10px;margin:0px;text-decoration:none;font-family:"overpass",sans-serif" href="https://www.redhat.com" target="_blank">Red Hat <span><br>
<br>
</span></a></p>
<span style="font-size:10px;margin:0px;color:rgb(153,153,153)">
<p style="font-size:10px;margin:0px">Milan,
Italy</p>
</span>
<p style="font-weight:normal;margin:0px 0px 6px;font-size:10px;color:rgb(153,153,153)"><span style="margin:0px;padding:0px">
<a style="color:rgb(0,136,206);font-size:10px;margin:0px;text-decoration:none;font-family:"overpass",sans-serif" href="mailto:vrigamon@redhat.com" target="_blank">vrigamon@redhat.com</a><br>
</span></p>
<p style="font-weight:normal;margin:0px 0px 6px;font-size:10px;color:rgb(153,153,153)"><span style="margin:0px;padding:0px">irc:
rigazilla </span> </p>
<a href="https://red.ht/sig" target="_blank"> <img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" height="auto" width="90"></a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="m_5548383007427065293m_-557338394167953243m_4400301142433856756mimeAttachmentHeader"></fieldset>
<br>
</div>
</div>
<pre>_______________________________________________
infinispan-dev mailing list
<a class="m_5548383007427065293m_-557338394167953243m_4400301142433856756moz-txt-link-abbreviated" href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a>
<a class="m_5548383007427065293m_-557338394167953243m_4400301142433856756moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a></pre>
</blockquote>
<p><br>
</p>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div class="m_5548383007427065293m_-557338394167953243gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<p style="font-weight:bold;margin:0;padding:0;font-size:14px;text-transform:uppercase;margin-bottom:0"><span>Vittorio</span>
<span>Rigamonti</span></p>
<p style="font-weight:normal;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><span>Senior
Software Engineer</span></p>
<p style="font-weight:normal;margin:0;font-size:10px;color:#999"><a style="color:#0088ce;font-size:10px;margin:0;text-decoration:none;font-family:'overpass',sans-serif" href="https://www.redhat.com" target="_blank">Red Hat <span><br>
<br>
</span></a></p>
<span style="font-size:10px;margin:0px;color:rgb(153,153,153)">
<p style="font-size:10px;margin:0">Milan, Italy</p>
</span>
<p style="font-weight:normal;margin:0px 0px 6px;font-size:10px;color:rgb(153,153,153)"><span style="margin:0px;padding:0px">
<a style="color:#0088ce;font-size:10px;margin:0;text-decoration:none;font-family:'overpass',sans-serif" href="mailto:vrigamon@redhat.com" target="_blank">vrigamon@redhat.com</a><br>
</span></p>
<p style="font-weight:normal;margin:0px 0px 6px;font-size:10px;color:rgb(153,153,153)"><span style="margin:0px;padding:0px">irc: rigazilla </span>
</p>
<a href="https://red.ht/sig" target="_blank"> <img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" height="auto" width="90"></a></div>
</div>
</div>
</div>
</div>
</blockquote>
<p><br>
</p>
</div>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a></blockquote></div></div></div>