<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Thanks for clarifying this Galder.<br>
Yes, the network layer is indeed the culprit and the purpose of
this experiment.<br>
<br>
What is the approach you envision regarding the IDL? Should we
strive for a pure IDL definition of the service? That could be an
interesting approach that would make it possible for a third party
to generate their own infinispan grpc client in any new language
that we do not already offer support, just based on the IDL. And
maybe using a different grpc implementation if they do not find
suitable the one from google.<br>
<br>
I was not suggesting we should do type transformation or anything
on the client side that would require an extra layer of code on
top of what grpc generates for the client, so maybe a pure IDL
based service definition would indeed be possible, without extra
helpers. No type transformation, just type information. Exposing
the type info that comes from the server would be enough, a lot
better than dumbing everything down to a byte[].<br>
<br>
Adrian<br>
<br>
On 05/30/2018 12:16 PM, Galder Zamarreno wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAAcPREvKyuqPYaevD8X-2j-rGX7+gP50dzEPTjWAkjs50s=n5A@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">On Tue, May 29, 2018 at 8:57 PM Adrian Nistor
<<a href="mailto:anistor@redhat.com" target="_blank"
moz-do-not-send="true">anistor@redhat.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div
class="m_5548383007427065293m_-557338394167953243moz-cite-prefix">Vittorio,
a few remarks regarding your statement "...The
alternative to this is to develop a protostream
equivalent for each supported language and it doesn't
seem really feasible to me."<br>
<br>
No way! That's a big misunderstanding. We do not need to
re-implement the protostream library in C/C++/C# or any
new supported language.<br>
Protostream is just for Java and it is compatible with
Google's protobuf lib we already use in the other
clients. We can continue using Google's protobuf lib for
these clients, with or without gRPC.<br>
Protostream does not handle protobuf services as gRPC
does, but we can add support for that with little
effort.<br>
<br>
The real problem here is if we want to replace our hot
rod invocation protocol with gRPC to save on the effort
of implementing and maintaining hot rod in all those
clients. I wonder why the obvious question is being
avoided in this thread.</div>
</div>
</blockquote>
<div><br>
</div>
</div>
<div dir="ltr">
<div class="gmail_quote">
<div>^ It is not being avoided. I stated it quite clearly
when I replied but maybe not with enough detail. So, I
said:</div>
</div>
</div>
<div dir="ltr">
<div class="gmail_quote">
<div><br>
</div>
<div>> The biggest problem I see in our client/server
architecture is the ability to quickly deliver
features/APIs across multiple language clients. Both
Vittorio and I have seen how long it takes to implement
all the different features available in Java client and
port them to Node.js, C/C++/C#...etc. This effort lead by
Vittorio is trying to improve on that by having some of
that work done for us. Granted, not all of it will be
done, but it should give us some good foundations on which
to build.</div>
<div><br>
</div>
<div>To expand on it a bit further: the reason it takes us
longer to get different features in is because each client
implements its own network layer, parses the protocol and
does type transformations (between byte[] and whatever the
client expects). </div>
<div><br>
</div>
<div>IMO, the most costly things there are getting the
network layer right (from experience with Node.js, it has
taken a while to do so) and parsing work (not only parsing
itself, but doing it in a efficient way). Network layer
also includes load balancing, failover, cluster
failover...etc.</div>
<div><br>
</div>
<div>From past experience, transforming from byte[] to what
the client expects has never really been very problematic
for me. What's been difficult here is coming up with
encoding architecture that Gustavo lead, whose aim was to
improve on the initial compatibility mode. But, with that
now clear, understood and proven to solve our issues, the
rest in this area should be fairly straightforward IMO.</div>
<div><br>
</div>
<div>Type transformation, once done, is a constant. As we
add more Hot Rod operations, it's mostly the parsing that
starts to become more work. Network can also become more
work if instead of RPC commands you start supporting
streams based commands.</div>
<div><br>
</div>
<div>gRPC solves the network (FYI: with key as HTTP header
and SubchannelPicker you can do hash-aware routing) and
parsing for us. I don't see the need for it to solve our
type transformations for us. If it does it, great, but
does it support our compatibility requirements? (I had
already told Vittorio to check Gustavo on this). Type
transformation is a lower prio for me, network and parsing
are more important.</div>
<div><br>
</div>
<div>Hope this clarifies better my POV.</div>
<div><br>
</div>
<div>Cheers</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div
class="m_5548383007427065293m_-557338394167953243moz-cite-prefix"><br>
<br>
Adrian</div>
</div>
<div text="#000000" bgcolor="#FFFFFF">
<div
class="m_5548383007427065293m_-557338394167953243moz-cite-prefix"><br>
<br>
On 05/29/2018 03:45 PM, Vittorio Rigamonti wrote:<br>
</div>
</div>
<div text="#000000" bgcolor="#FFFFFF">
<blockquote type="cite">
<div dir="ltr">
<div>
<div>Thanks Adrian,<br>
<br>
</div>
of course there's a marshalling work under the
cover and that is reflected into the generated
code (specially the accessor methods generated
from the oneof clause).<br>
<br>
</div>
<div>My opinion is that on the client side this
could be accepted, as long as the API are well
defined and documented: application developer can
build an adhoc decorator on the top if needed. The
alternative to this is to develop a protostream
equivalent for each supported language and it
doesn't seem really feasible to me.<br>
<br>
</div>
<div>On the server side (java only) the situation is
different: protobuf is optimized for streaming not
for storing so probably a Protostream layer is
needed.<br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, May 28, 2018 at
4:47 PM, Adrian Nistor <span dir="ltr"><<a
href="mailto:anistor@redhat.com"
target="_blank" moz-do-not-send="true">anistor@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div
class="m_5548383007427065293m_-557338394167953243m_4400301142433856756moz-cite-prefix">Hi
Vittorio,<br>
thanks for exploring gRPC. It seems like a
very elegant solution for exposing services.
I'll have a look at your PoC soon.<br>
<br>
I feel there are some remarks that need to
be made regarding gRPC. gRPC is just some
nice cheesy topping on top of protobuf.
Google's implementation of protobuf, to be
more precise. <br>
It does not need handwritten marshallers,
but the 'No need for marshaller' does not
accurately describe it. Marshallers are
needed and are generated under the cover by
the library and so are the data objects and
you are unfortunately forced to use them.
That's both the good news and the bad news:)
The whole thing looks very promising and
friendly for many uses cases, especially for
demos and PoCs :))). Nobody wants to write
those marshallers. But it starts to become a
nuisance if you want to use your own data
objects.<br>
There is also the ugliness and excessive
memory footprint of the generated code,
which is the reason Infinispan did not adopt
the protobuf-java library although it did
adopt protobuf as an encoding format.<br>
The Protostream library was created as an
alternative implementation to solve the
aforementioned problems with the generated
code. It solves this by letting the user
provide their own data objects. And for the
marshallers it gives you two options: a)
write the marshaller yourself (hated), b)
annotated your data objects and the
marshaller gets generated (loved).
Protostream does not currently support
service definitions right now but this is
something I started to investigate recently
after Galder asked me if I think it's
doable. I think I'll only find out after I
do it:)<br>
<br>
Adrian
<div>
<div
class="m_5548383007427065293m_-557338394167953243h5"><br>
<br>
On 05/28/2018 04:15 PM, Vittorio
Rigamonti wrote:<br>
</div>
</div>
</div>
<blockquote type="cite">
<div>
<div
class="m_5548383007427065293m_-557338394167953243h5">
<div dir="ltr">
<div>Hi Infinispan developers,<br>
<br>
</div>
<div>I'm working on a solution for
developers who need to access
Infinispan services through
different programming languages.<br>
</div>
<div><br>
The focus is not on developing a
full featured client, but rather
discover the value and the limits of
this approach.<br>
<div><br>
</div>
</div>
<div>- is it possible to automatically
generate useful clients in different
languages?<br>
</div>
<div>- can that clients interoperate
on the same cache with the same data
types?<br>
</div>
<div><br>
</div>
<span
id="m_5548383007427065293m_-557338394167953243m_4400301142433856756m_-8148977098275501500gmail-result_box"
class="m_5548383007427065293m_-557338394167953243m_4400301142433856756m_-8148977098275501500gmail-"
lang="en"><span
class="m_5548383007427065293m_-557338394167953243m_4400301142433856756m_-8148977098275501500gmail-">I
came out with a small prototype
that I would like to submit to you
and on which I would like to
gather your impressions.</span></span><br>
<div><br>
You can found the project here [1]:
is a gRPC-based client/server
architecture for Infinispan based on
and EmbeddedCache, with very few
features exposed atm.<br>
</div>
<div><br>
Currently the project is nothing
more than a poc with the following
interesting features:<br>
<br>
</div>
<div>- client can be generated in all
the grpc supported language: java,
go, c++ examples are provided;<br>
</div>
<div>- the interface is full typed. No
need for marshaller and clients
build in different language can
cooperate on the same cache;<br>
</div>
<div><br>
</div>
<div>The second item is my preferred
one beacuse it frees the developer
from data marshalling.<br>
<br>
</div>
<div>What do you think about?<br>
</div>
<div>Sounds interesting?<br>
</div>
<div>Can you see any flaw?<br>
</div>
<div><br>
There's also a list of issues for
the future [2], basically I would
like to investigate these questions:<br>
How far this architecture can go?<br>
Topology, events, queries... how
many of the Infinispan features can
be fit in a grpc architecture?<br>
</div>
<div><br>
</div>
<div>Thank you<br>
</div>
<div>Vittorio<br>
</div>
<br>
[1] <a
href="https://github.com/rigazilla/ispn-grpc"
target="_blank"
moz-do-not-send="true">https://github.com/rigazilla/ispn-grpc</a><br
clear="all">
<div>
<div>
<div>[2] <a
href="https://github.com/rigazilla/ispn-grpc/issues"
target="_blank"
moz-do-not-send="true">https://github.com/rigazilla/ispn-grpc/issues</a><br
clear="all">
<br>
</div>
<div>-- <br>
<div
class="m_5548383007427065293m_-557338394167953243m_4400301142433856756m_-8148977098275501500gmail-m_-8431198291286624134m_1982721755148179188gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<p
style="font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><span>Vittorio</span>
<span>Rigamonti</span></p>
<p
style="font-weight:normal;font-size:10px;margin:0px
0px
4px;text-transform:uppercase"><span>Senior
Software Engineer</span></p>
<p
style="font-weight:normal;margin:0px;font-size:10px;color:rgb(153,153,153)"><a
style="color:rgb(0,136,206);font-size:10px;margin:0px;text-decoration:none;font-family:"overpass",sans-serif"
href="https://www.redhat.com" target="_blank" moz-do-not-send="true">Red
Hat <span><br>
<br>
</span></a></p>
<span
style="font-size:10px;margin:0px;color:rgb(153,153,153)">
<p
style="font-size:10px;margin:0px">Milan,
Italy</p>
</span>
<p
style="font-weight:normal;margin:0px
0px
6px;font-size:10px;color:rgb(153,153,153)"><span
style="margin:0px;padding:0px"> <a
style="color:rgb(0,136,206);font-size:10px;margin:0px;text-decoration:none;font-family:"overpass",sans-serif"
href="mailto:vrigamon@redhat.com" target="_blank" moz-do-not-send="true">vrigamon@redhat.com</a><br>
</span></p>
<p
style="font-weight:normal;margin:0px
0px
6px;font-size:10px;color:rgb(153,153,153)"><span
style="margin:0px;padding:0px">irc: rigazilla </span> </p>
<a
href="https://red.ht/sig"
target="_blank"
moz-do-not-send="true">
<img
src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png"
moz-do-not-send="true" height="auto" width="90"></a></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset
class="m_5548383007427065293m_-557338394167953243m_4400301142433856756mimeAttachmentHeader"></fieldset>
<br>
</div>
</div>
<pre>_______________________________________________
infinispan-dev mailing list
<a class="m_5548383007427065293m_-557338394167953243m_4400301142433856756moz-txt-link-abbreviated" href="mailto:infinispan-dev@lists.jboss.org" target="_blank" moz-do-not-send="true">infinispan-dev@lists.jboss.org</a>
<a class="m_5548383007427065293m_-557338394167953243m_4400301142433856756moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank" moz-do-not-send="true">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a></pre>
</blockquote>
<p><br>
</p>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div
class="m_5548383007427065293m_-557338394167953243gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<p
style="font-weight:bold;margin:0;padding:0;font-size:14px;text-transform:uppercase;margin-bottom:0"><span>Vittorio</span>
<span>Rigamonti</span></p>
<p
style="font-weight:normal;font-size:10px;margin:0px
0px 4px;text-transform:uppercase"><span>Senior
Software Engineer</span></p>
<p
style="font-weight:normal;margin:0;font-size:10px;color:#999"><a
style="color:#0088ce;font-size:10px;margin:0;text-decoration:none;font-family:'overpass',sans-serif"
href="https://www.redhat.com"
target="_blank" moz-do-not-send="true">Red
Hat <span><br>
<br>
</span></a></p>
<span
style="font-size:10px;margin:0px;color:rgb(153,153,153)">
<p style="font-size:10px;margin:0">Milan,
Italy</p>
</span>
<p style="font-weight:normal;margin:0px 0px
6px;font-size:10px;color:rgb(153,153,153)"><span
style="margin:0px;padding:0px"> <a
style="color:#0088ce;font-size:10px;margin:0;text-decoration:none;font-family:'overpass',sans-serif"
href="mailto:vrigamon@redhat.com"
target="_blank" moz-do-not-send="true">vrigamon@redhat.com</a><br>
</span></p>
<p style="font-weight:normal;margin:0px 0px
6px;font-size:10px;color:rgb(153,153,153)"><span
style="margin:0px;padding:0px">irc:
rigazilla </span> </p>
<a href="https://red.ht/sig" target="_blank"
moz-do-not-send="true"> <img
src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png"
moz-do-not-send="true" height="auto"
width="90"></a></div>
</div>
</div>
</div>
</div>
</blockquote>
<p><br>
</p>
</div>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org"
target="_blank" moz-do-not-send="true">infinispan-dev@lists.jboss.org</a><br>
<a
href="https://lists.jboss.org/mailman/listinfo/infinispan-dev"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a></blockquote>
</div>
</div>
</div>
<!--'"--><br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
infinispan-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/infinispan-dev">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a></pre>
</blockquote>
<p><br>
</p>
</body>
</html>