3:08 a.m.
On 12/12/2018 03:30 PM, Sebastian Laskawiec wrote:
On Tue, Dec 11, 2018 at 11:13 AM Radim Vansa <rvansa(a)redhat.com
<mailto:rvansa@redhat.com>> wrote:
I dislike having any logic based on the port number in some range;
it's
not common that behaviour would change if you set port to 9xxx
instead
of 8xxx.
That's not a problem with my approach, since you can always manually
turn the setting off or on. Here's how you do it:
ConfigurationBuilder cb = ...
cb.singlePort(SinglePortMode.ENABLED); // other options: DISABLED and AUTO
Adding config options is just a way to avoid solving problems :)
Remember the famous quote: "Less knobs!"
Is there an (up-to-date) design doc?
No, this is just a proposal. I was hoping that you guys like it and
then, with some thumbs up, I could update the design doc.
Here's the most up-to-date version in case you were looking for it:
https://github.com/infinispan/infinispan-designs/blob/master/Single_port....
I don't fully follow, but if there's a problem in the HTTP
handlers you
can add a PING-detecting handler below...?
Thanks for the hint Radim!
Inspired by your idea I went ahead and checked how OpenShift Router
behaves. It turns out that it responds HTTP 400 if you throw Hotrod
bytes at it and then drops the connection.
I understand that the reason to have Hot Rod PING sent as the first
operation is to make sure that a new client that tries to connect to old
server won't confuse the server, is that correct? Or is there anything else?
I'll assume that a new server will handle both Hot Rod PING and HTTP
request correctly without any prelude, and old one will be ok with Hot
Rod PING only. I don't really understand the:
implementing it this way seems a bit "inconvenient" to me.
The Ping
Operation uses 60s timeout, which seems to be a good fit as a default.
Unfortunately, for the Single Port functionality, this means we'd need
to wait 60s until we try to send HTTP request and do an upgrade
why would you wait for 60 seconds? If the other end is Infinispan server
(old or new), you just send HR PING and you're done, server will proceed
correctly. If the other server is a router, you'll get a response
starting with 'HTTP': in Hot Rod protocol that would be parsed as opCode
0x54 which is illegal response code (the id belongs to
COUNTER_RESET_REQUEST). At this point you know that this connection is
going to be closed, and can immediately start another one (is *this* the
problem?) that will send a HTTP request with Upgrade header.
I also realized, there's one more moving bit - TCP Keepalive.
Luckily,
we can control this setting over configuration in our standalone.xml.
However, it is perfectly legal what I've seen in Netty (do not respond
and keep the connection alive assuming that TCP Keepalive is set to true).
The more I think about this, the more I'm convinced that the Single
Port support should be explicitly set in the client (or inferred from
the configuration). I do not know how Nginx, Linkerd or Envoy behaves
in situation when they expect HTTP and get a stream of bytes. Relying
on this partially unknown behavior for doing our upgrade procedure
doesn't seem right to me.
FYI Envoy does the same, send 400 and terminate the connection.
Just in case you're worried about the additional logic on the client
side - it's super small. Really, only 13 lines including brackets ;)
https://github.com/infinispan/infinispan/pull/6133/files#diff-684a10c939f...
I am not worried about any logic in the client code, I am worried about
logic between chair and keyboard.
R.
Radim
On 12/10/2018 03:27 PM, Sebastian Laskawiec wrote:
> Hey guys,
>
> During Infinispan F2F, I had a short discussion with Tristan on
Single
> Port client-side implementation. Back then, we agreed that the
client
> should always send a Hotrod Ping request and if won't get any
response
> (or get some HTTP content back), it will try to upgrade to the
Hotrod
> protocol using Single Port.
>
> I've been playing with the implementation for a while, and
> implementing it this way seems a bit "inconvenient" to me. The Ping
> Operation uses 60s timeout, which seems to be a good fit as a
default.
> Unfortunately, for the Single Port functionality, this means
we'd need
> to wait 60s until we try to send HTTP request and do an upgrade.
Also,
> another problematic part is in Netty's HTTP handlers
> (HttpObjectDecoder, HttpServerCodec and ByteToMessageDecoder). When
> those classes fail to decode a message (REST expects HTTP rather
than
> a stream of bytes specific to Hotrod protocol), they just ignore it
> and keep the channel in active state (which also makes sense for
> HTTP/1.1 and HTTP/2).
>
> At this point, my intuition tells, that this doesn't look right and
> seems to be a over-complicated. The whole HTTP upgrade idea
seems to
> work the other way around, use HTTP as a fallback and then
upgrade to
> other protocols. Forcing it to work a bit differently requires some
> more effort.
>
> What if we preserved the Single Port setting in the client
> configuration but implemented it as an enum with the following
values
> - true/false/auto. In automatic mode, the client would check if the
> server port is set to 8\d{1,3} (this covers 80, 8080, 8081, 8443
and
> friends). If that is true, we'd try to follow HTTP Upgrade
procedure.
> This looks very simple and I think this might actually work. Please
> note, that we need the single port setting in the client
configuration
> to cover some corner cases like the Single Port exposed on
different
> port (like 4444) or Hot Rod exposed on port that starts with 8.
>
> What do you think about such simplification?
>
> Thanks,
> Sebastian
>
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
<mailto:infinispan-dev@lists.jboss.org>
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa <rvansa(a)redhat.com <mailto:rvansa@redhat.com>>
JBoss Performance Team
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org>
https://lists.jboss.org/mailman/listinfo/infinispan-dev
_______________________________________________
infinispan-dev mailing list
infinispan-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa <rvansa(a)redhat.com>
JBoss Performance Team