3:35 a.m.
Hi all,
@Vittorio, thanks a lot for working on this!
Let me explain some of the background behind this effort so that we're all
on the same page:
The biggest problem I see in our client/server architecture is the ability
to quickly deliver features/APIs across multiple language clients. Both
Vittorio and I have seen how long it takes to implement all the different
features available in Java client and port them to Node.js, C/C++/C#...etc.
This effort lead by Vittorio is trying to improve on that by having some of
that work done for us. Granted, not all of it will be done, but it should
give us some good foundations on which to build.
One thing I mentioned to Vittorio is that he should investigate what the
performance impact of using gRPC is. This is crucial to decide whether to
take this forward or not. This should really have been done by now so that
other devs are aware of the cost in terms of latency and memory
consumption. As you can see from the first comment, there are already
concerns with its memory consumption. So, this needs to be done ASAP so
that we're aware of the consequences right away.
Also, when I looked at gRPC, I was considering having the base layer use
only bytes, and we'd build the marshallers/encoders...etc we need on top.
Maybe both approaches can be compared from the POV of performance.
If gRPC performance is not up to scratch, we have the contacts to see if
things can be improved.
Once again, as I mentioned to Vittorio separately, if we can't rely on
gRPC (or similar tool), it'd be nice to have just a C client (or a more
typesafe client that compiles into C, e.g. Rust) that uses protobuf
serialized messages and get any other language to be a wrapper of that.
This is possible with Node.js and Haskell for example. With Java this is
not currently an option since JNI is slow and cumbersome but maybe with
Project Panama [4] this won't be problem in the future. So maybe a Java (w/
Netty) and C clients and the rest interfacing to them would be the way if
gRPC does not work out.
Cheers
On Mon, May 28, 2018 at 4:50 PM Adrian Nistor <anistor(a)redhat.com> wrote:
Hi Vittorio,
thanks for exploring gRPC. It seems like a very elegant solution for
exposing services. I'll have a look at your PoC soon.
I feel there are some remarks that need to be made regarding gRPC. gRPC is
just some nice cheesy topping on top of protobuf. Google's implementation
of protobuf, to be more precise.
It does not need handwritten marshallers, but the 'No need for marshaller'
does not accurately describe it. Marshallers are needed and are generated
under the cover by the library and so are the data objects and you are
unfortunately forced to use them. That's both the good news and the bad
news:) The whole thing looks very promising and friendly for many uses
cases, especially for demos and PoCs :))). Nobody wants to write those
marshallers. But it starts to become a nuisance if you want to use your own
data objects.
There is also the ugliness and excessive memory footprint of the generated
code, which is the reason Infinispan did not adopt the protobuf-java
library although it did adopt protobuf as an encoding format.
The Protostream library was created as an alternative implementation to
solve the aforementioned problems with the generated code. It solves this
by letting the user provide their own data objects. And for the marshallers
it gives you two options: a) write the marshaller yourself (hated), b)
annotated your data objects and the marshaller gets generated (loved).
Protostream does not currently support service definitions right now but
this is something I started to investigate recently after Galder asked me
if I think it's doable. I think I'll only find out after I do it:)
Adrian
On 05/28/2018 04:15 PM, Vittorio Rigamonti wrote:
Hi Infinispan developers,
I'm working on a solution for developers who need to access Infinispan
services through different programming languages.
The focus is not on developing a full featured client, but rather discover
the value and the limits of this approach.
- is it possible to automatically generate useful clients in different
languages?
- can that clients interoperate on the same cache with the same data types?
I came out with a small prototype that I would like to submit to you and
on which I would like to gather your impressions.
You can found the project here [1]: is a gRPC-based client/server
architecture for Infinispan based on and EmbeddedCache, with very few
features exposed atm.
Currently the project is nothing more than a poc with the following
interesting features:
- client can be generated in all the grpc supported language: java, go,
c++ examples are provided;
- the interface is full typed. No need for marshaller and clients build in
different language can cooperate on the same cache;
The second item is my preferred one beacuse it frees the developer from
data marshalling.
What do you think about?
Sounds interesting?
Can you see any flaw?
There's also a list of issues for the future [2], basically I would like
to investigate these questions:
How far this architecture can go?
Topology, events, queries... how many of the Infinispan features can be
fit in a grpc architecture?
Thank you
Vittorio
[1] https://github.com/rigazilla/ispn-grpc
[2] https://github.com/rigazilla/ispn-grpc/issues
--
Vittorio Rigamonti
Senior Software Engineer
Red Hat
<https://www.redhat.com>
Milan, Italy
vrigamon(a)redhat.com
irc: rigazilla
<https://red.ht/sig>
_______________________________________________
infinispan-dev mailing
listinfinispan-dev@lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev