<div dir="ltr"><div>Thanks Galder,<br><br></div>comments inline<br><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 29, 2018 at 10:35 AM, Galder Zamarreno <span dir="ltr"><<a href="mailto:galder@redhat.com" target="_blank">galder@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi all,<div><br></div><div>@Vittorio, thanks a lot for working on this!</div><div><br></div><div>Let me explain some of the background behind this effort so that we're all on the same page:</div><div><br></div><div>The biggest problem I see in our client/server architecture is the ability to quickly deliver features/APIs across multiple language clients. Both Vittorio and I have seen how long it takes to implement all the different features available in Java client and port them to Node.js, C/C++/C#...etc. This effort lead by Vittorio is trying to improve on that by having some of that work done for us. Granted, not all of it will be done, but it should give us some good foundations on which to build.</div><div><br></div><div>One thing I mentioned to Vittorio is that he should investigate what the performance impact of using gRPC is. This is crucial to decide whether to take this forward or not. This should really have been done by now so that other devs are aware of the cost in terms of latency and memory consumption. As you can see from the first comment, there are already concerns with its memory consumption. So, this needs to be done ASAP so that we're aware of the consequences right away.</div></div></blockquote><div> </div><div>we need to define some scenarios to be sure to collect meaningful data. Well probably with memory we can do some quick estimation.<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div> </div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>Also, when I looked at gRPC, I was considering having the base layer use only bytes, and we'd build the marshallers/encoders...etc we need on top. Maybe both approaches can be compared from the POV of performance.</div><div><br></div><div>If gRPC performance is not up to scratch, we have the contacts to see if things can be improved.</div><div><br></div><div>Once again, as I mentioned to Vittorio separately, if we can't rely on gRPC (or similar tool), it'd be nice to have just a C client (or a more typesafe client that compiles into C, e.g. Rust) that uses protobuf serialized messages and get any other language to be a wrapper of that. This is possible with Node.js and Haskell for example. With Java this is not currently an option since JNI is slow and cumbersome but maybe with Project Panama [4] this won't be problem in the future. So maybe a Java (w/ Netty) and C clients and the rest interfacing to them would be the way if gRPC does not work out.</div></div></blockquote><div><br></div><div>i did some experiments based on SWIG here:<br><a href="https://github.com/rigazilla/hotswig" target="_blank">https://github.com/rigazilla/h<wbr>otswig</a><br><br></div><div>the c/wrapper architecture works quite well with the simple get/put use case, things become harder with events, queries...<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Cheers</div><div><div class="m_-3262568369977296635m_-8330555892289882607gmail-h5"><br><div class="gmail_quote"><div dir="ltr">On Mon, May 28, 2018 at 4:50 PM Adrian Nistor <<a href="mailto:anistor@redhat.com" target="_blank">anistor@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div class="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900moz-cite-prefix">Hi Vittorio,<br>
thanks for exploring gRPC. It seems like a very elegant solution
for exposing services. I'll have a look at your PoC soon.<br>
<br>
I feel there are some remarks that need to be made regarding gRPC.
gRPC is just some nice cheesy topping on top of protobuf. Google's
implementation of protobuf, to be more precise. <br>
It does not need handwritten marshallers, but the 'No need for
marshaller' does not accurately describe it. Marshallers are
needed and are generated under the cover by the library and so are
the data objects and you are unfortunately forced to use them.
That's both the good news and the bad news:) The whole thing looks
very promising and friendly for many uses cases, especially for
demos and PoCs :))). Nobody wants to write those marshallers. But
it starts to become a nuisance if you want to use your own data
objects.<br>
There is also the ugliness and excessive memory footprint of the
generated code, which is the reason Infinispan did not adopt the
protobuf-java library although it did adopt protobuf as an
encoding format.<br>
The Protostream library was created as an alternative
implementation to solve the aforementioned problems with the
generated code. It solves this by letting the user provide their
own data objects. And for the marshallers it gives you two
options: a) write the marshaller yourself (hated), b) annotated
your data objects and the marshaller gets generated (loved).
Protostream does not currently support service definitions right
now but this is something I started to investigate recently after
Galder asked me if I think it's doable. I think I'll only find out
after I do it:)</div></div><div bgcolor="#FFFFFF"><div class="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900moz-cite-prefix"><br>
<br>
Adrian</div></div><div bgcolor="#FFFFFF"><div class="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900moz-cite-prefix"><br>
<br>
On 05/28/2018 04:15 PM, Vittorio Rigamonti wrote:<br>
</div></div><div bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">
<div>Hi Infinispan developers,<br>
<br>
</div>
<div>I'm working on a solution for developers who need to access
Infinispan services through different programming languages.<br>
</div>
<div><br>
The focus is not on developing a full featured client, but
rather discover the value and the limits of this approach.<br>
<div><br>
</div>
</div>
<div>- is it possible to automatically generate useful clients
in different languages?<br>
</div>
<div>- can that clients interoperate on the same cache with the
same data types?<br>
</div>
<div><br>
</div>
<span id="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900m_-8148977098275501500gmail-result_box" class="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900m_-8148977098275501500gmail-" lang="en"><span class="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900m_-8148977098275501500gmail-">I came out with a small
prototype that I would like to submit to you and on which I
would like to gather your impressions.</span></span><br>
<div><br>
You can found the project here [1]: is a gRPC-based
client/server architecture for Infinispan based on and
EmbeddedCache, with very few features exposed atm.<br>
</div>
<div><br>
Currently the project is nothing more than a poc with the
following interesting features:<br>
<br>
</div>
<div>- client can be generated in all the grpc supported
language: java, go, c++ examples are provided;<br>
</div>
<div>- the interface is full typed. No need for marshaller and
clients build in different language can cooperate on the same
cache;<br>
</div>
<div><br>
</div>
<div>The second item is my preferred one beacuse it frees the
developer from data marshalling.<br>
<br>
</div>
<div>What do you think about?<br>
</div>
<div>Sounds interesting?<br>
</div>
<div>Can you see any flaw?<br>
</div>
<div><br>
There's also a list of issues for the future [2], basically I
would like to investigate these questions:<br>
How far this architecture can go?<br>
Topology, events, queries... how many of the Infinispan
features can be fit in a grpc architecture?<br>
</div>
<div><br>
</div>
<div>Thank you<br>
</div>
<div>Vittorio<br>
</div>
<br>
[1] <a href="https://github.com/rigazilla/ispn-grpc" target="_blank">https://github.com/rigazilla/i<wbr>spn-grpc</a><br clear="all">
<div>
<div>
<div>[2] <a href="https://github.com/rigazilla/ispn-grpc/issues" target="_blank">https://github.com/rigazilla/i<wbr>spn-grpc/issues</a><br clear="all">
<br>
</div>
<div>-- <br>
<div class="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900m_-8148977098275501500gmail-m_-8431198291286624134m_1982721755148179188gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<p style="font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><span>Vittorio</span>
<span>Rigamonti</span></p>
<p style="font-weight:normal;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><span>Senior
Software Engineer</span></p>
<p style="font-weight:normal;margin:0px;font-size:10px;color:rgb(153,153,153)"><a style="color:rgb(0,136,206);font-size:10px;margin:0px;text-decoration:none;font-family:"overpass",sans-serif" href="https://www.redhat.com" target="_blank">Red Hat <span><br>
<br>
</span></a></p>
<span style="font-size:10px;margin:0px;color:rgb(153,153,153)">
<p style="font-size:10px;margin:0px">Milan,
Italy</p>
</span>
<p style="font-weight:normal;margin:0px 0px 6px;font-size:10px;color:rgb(153,153,153)"><span style="margin:0px;padding:0px">
<a style="color:rgb(0,136,206);font-size:10px;margin:0px;text-decoration:none;font-family:"overpass",sans-serif" href="mailto:vrigamon@redhat.com" target="_blank">vrigamon@redhat.com</a><br>
</span></p>
<p style="font-weight:normal;margin:0px 0px 6px;font-size:10px;color:rgb(153,153,153)"><span style="margin:0px;padding:0px">irc: rigazilla </span>
</p>
<a href="https://red.ht/sig" target="_blank"> <img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" height="auto" width="90"></a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
infinispan-dev mailing list
<a class="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900moz-txt-link-abbreviated" href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a>
<a class="m_-3262568369977296635m_-8330555892289882607gmail-m_-62534169007525770m_-3218386508718938900moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailma<wbr>n/listinfo/infinispan-dev</a></pre>
</blockquote>
<p><br>
</p>
</div>
______________________________<wbr>_________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org" target="_blank">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" rel="noreferrer" target="_blank">https://lists.jboss.org/mailma<wbr>n/listinfo/infinispan-dev</a></blockquote></div></div></div></div>
</blockquote></div><br><br clear="all"><br>-- <br><div class="m_-3262568369977296635m_-8330555892289882607gmail_signature"><div dir="ltr"><div><div dir="ltr"><p style="font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><span>Vittorio</span> <span>Rigamonti</span></p>
<p style="font-weight:normal;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><span>Senior Software Engineer</span></p>
<p style="font-weight:normal;margin:0px;font-size:10px;color:rgb(153,153,153)"><a style="color:rgb(0,136,206);font-size:10px;margin:0px;text-decoration:none;font-family:"overpass",sans-serif" href="https://www.redhat.com" target="_blank">Red Hat <span><br><br></span></a></p>
<span style="font-size:10px;margin:0px;color:rgb(153,153,153)"><p style="font-size:10px;margin:0px">Milan, Italy</p></span>
<p style="font-weight:normal;margin:0px 0px 6px;font-size:10px;color:rgb(153,153,153)"><span style="margin:0px;padding:0px">
<a style="color:rgb(0,136,206);font-size:10px;margin:0px;text-decoration:none;font-family:"overpass",sans-serif" href="mailto:vrigamon@redhat.com" target="_blank">vrigamon@redhat.com</a><br></span></p><p style="font-weight:normal;margin:0px 0px 6px;font-size:10px;color:rgb(153,153,153)"><span style="margin:0px;padding:0px">irc: rigazilla </span>
</p>
<a href="https://red.ht/sig" target="_blank"> <img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" height="auto" width="90"></a></div></div></div></div>
</div></div></div></div>