[jboss-jira] [JBoss JIRA] (WFLY-11639) mod cluster and HTTP2 enabled (the default) not working
Andreas Asplund (Jira)
issues at jboss.org
Wed Jan 23 12:26:00 EST 2019
[ https://issues.jboss.org/browse/WFLY-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andreas Asplund updated WFLY-11639:
-----------------------------------
Description:
mod cluster and HTTP2 enabled (the default) when using listener default in the modcluster subsystem. When turning off HTTP2 the clustering starts working again. Nothing is printed in the logs by default so DEBUG logging has to be turned on for undertow.
Steps to reproduce:
{code}
unzip wildfly-dist-15.0.1.Final.zip
# Terminal 1
cd wildfly-15.0.1.Final/bin/
./standalone.sh -c standalone-load-balancer.xml -Djava.net.preferIPv4Stack=true -b 127.0.0.1 -Djboss.modcluster.multicast.address=230.0.0.4
# Terminal 2
cd wildfly-15.0.1.Final/bin/
./standalone.sh -c standalone-ha.xml -Djboss.node.name=node1 -Djboss.socket.binding.port-offset=100 -Djboss.modcluster.multicast.address=230.0.0.4
# Terminal 3
cd wildfly-15.0.1.Final/bin/
./standalone.sh -c standalone-ha.xml -Djboss.node.name=node2 -Djboss.socket.binding.port-offset=200 -Djboss.modcluster.multicast.address=230.0.0.4
# Terminal 4
cd wildfly-15.0.1.Final/bin/
./jboss-cli.sh -c
/subsystem=logging/logger=io.undertow:add(level=ALL)
/subsystem=logging/console-handler=CONSOLE:write-attribute(name=level, value=ALL)
connect localhost:10090
/subsystem=logging/logger=io.undertow:add(level=ALL)
/subsystem=logging/console-handler=CONSOLE:write-attribute(name=level, value=ALL)
/subsystem=modcluster/proxy=default:write-attribute(name=listener,value=default)
reload
connect localhost:10190
/subsystem=logging/logger=io.undertow:add(level=ALL)
/subsystem=logging/console-handler=CONSOLE:write-attribute(name=level, value=ALL)
/subsystem=modcluster/proxy=default:write-attribute(name=listener,value=default)
reload
{code}
The following can be seen in the logs on the proxied servers:
{code}
16:30:23,409 TRACE [io.undertow.request] (default I/O-7) Opened connection with /127.0.0.1:50002
16:30:23,410 DEBUG [io.undertow.request.error-response] (default I/O-7) Setting error code 500 for exchange HttpServerExchange{ OPTIONS * request {HTTP2-Settings=[AAEAABAAAAIAAAABAAQAAP//AAUAAEAA], Connection=[Upgrade, HTTP2-Settings], Upgrade=[h2c], User-Agent=[mod_cluster ping], Host=[127.0.0.1]} response {}}: java.lang.RuntimeException
at io.undertow.server.HttpServerExchange.setStatusCode(HttpServerExchange.java:1410)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393)
at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
16:30:23,410 DEBUG [io.undertow.request.io] (default I/O-7) UT005013: An IOException occurred: java.io.IOException: Invalid base64 character encountered: 47
at io.undertow.util.FlexBase64$Decoder.nextByte(FlexBase64.java:1048)
at io.undertow.util.FlexBase64$Decoder.nextByte(FlexBase64.java:1015)
at io.undertow.util.FlexBase64$Decoder.decode(FlexBase64.java:1277)
at io.undertow.util.FlexBase64$Decoder.decode(FlexBase64.java:1347)
at io.undertow.util.FlexBase64$Decoder.decode(FlexBase64.java:1413)
at io.undertow.util.FlexBase64$Decoder.access$500(FlexBase64.java:983)
at io.undertow.util.FlexBase64.decodeURL(FlexBase64.java:320)
at io.undertow.server.protocol.http2.Http2UpgradeHandler.handleHttp2Upgrade(Http2UpgradeHandler.java:158)
at io.undertow.server.protocol.http2.Http2UpgradeHandler.handleUpgradeBody(Http2UpgradeHandler.java:107)
at io.undertow.server.protocol.http2.Http2UpgradeHandler.handleRequest(Http2UpgradeHandler.java:97)
at io.undertow.server.handlers.DisallowedMethodsHandler.handleRequest(DisallowedMethodsHandler.java:61)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360)
at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
16:30:23,411 TRACE [io.undertow.server.HttpServerExchange] (default I/O-7) Starting to write response for HttpServerExchange{ OPTIONS * request {HTTP2-Settings=[AAEAABAAAAIAAAABAAQAAP//AAUAAEAA], Connection=[Upgrade, HTTP2-Settings], Upgrade=[h2c], User-Agent=[mod_cluster ping], Host=[127.0.0.1]} response {Connection=[keep-alive], Content-Length=[0], Date=[Wed, 23 Jan 2019 15:30:23 GMT]}}
{code}
was:
mod cluster and HTTP2 enabled (the default) when using listener default in the modcluster subsystem. When turning off HTTP2 the clustering starts working again. Nothing is printed in the logs by default so DEBUG logging has to be turned on for undertow.
Steps to reproduce:
{code}
unzip wildfly-dist-15.0.1.Final.zip
# Terminal 1
cd wildfly-15.0.1.Final/bin/
./standalone.sh -c standalone-load-balancer.xml -Djava.net.preferIPv4Stack=true -b 127.0.0.1 -Djboss.modcluster.multicast.address=230.0.0.4
# Terminal 2
cd wildfly-15.0.1.Final/bin/
./standalone.sh -c standalone-ha.xml -Djboss.node.name=node1 -Djboss.socket.binding.port-offset=100 -Djboss.modcluster.multicast.address=230.0.0.4
# Terminal 3
cd wildfly-15.0.1.Final/bin/
./standalone.sh -c standalone-ha.xml -Djboss.node.name=node2 -Djboss.socket.binding.port-offset=200 -Djboss.modcluster.multicast.address=230.0.0.4
# Terminal 4
cd wildfly-15.0.1.Final/bin/
./jboss-cli.sh -c
/subsystem=logging/logger=io.undertow:add(level=ALL)
/subsystem=logging/console-handler=CONSOLE:write-attribute(name=level, value=ALL)
connect localhost:10090
/subsystem=logging/logger=io.undertow:add(level=ALL)
/subsystem=logging/console-handler=CONSOLE:write-attribute(name=level, value=ALL)
/subsystem=modcluster/proxy=default:write-attribute(name=listener,value=default)
reload
connect localhost:10190
/subsystem=logging/logger=io.undertow:add(level=ALL)
/subsystem=logging/console-handler=CONSOLE:write-attribute(name=level, value=ALL)
/subsystem=modcluster/proxy=default:write-attribute(name=listener,value=default)
reload
{code}
The following can be seen in the logs:
{code}
16:30:23,409 TRACE [io.undertow.request] (default I/O-7) Opened connection with /127.0.0.1:50002
16:30:23,410 DEBUG [io.undertow.request.error-response] (default I/O-7) Setting error code 500 for exchange HttpServerExchange{ OPTIONS * request {HTTP2-Settings=[AAEAABAAAAIAAAABAAQAAP//AAUAAEAA], Connection=[Upgrade, HTTP2-Settings], Upgrade=[h2c], User-Agent=[mod_cluster ping], Host=[127.0.0.1]} response {}}: java.lang.RuntimeException
at io.undertow.server.HttpServerExchange.setStatusCode(HttpServerExchange.java:1410)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393)
at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
16:30:23,410 DEBUG [io.undertow.request.io] (default I/O-7) UT005013: An IOException occurred: java.io.IOException: Invalid base64 character encountered: 47
at io.undertow.util.FlexBase64$Decoder.nextByte(FlexBase64.java:1048)
at io.undertow.util.FlexBase64$Decoder.nextByte(FlexBase64.java:1015)
at io.undertow.util.FlexBase64$Decoder.decode(FlexBase64.java:1277)
at io.undertow.util.FlexBase64$Decoder.decode(FlexBase64.java:1347)
at io.undertow.util.FlexBase64$Decoder.decode(FlexBase64.java:1413)
at io.undertow.util.FlexBase64$Decoder.access$500(FlexBase64.java:983)
at io.undertow.util.FlexBase64.decodeURL(FlexBase64.java:320)
at io.undertow.server.protocol.http2.Http2UpgradeHandler.handleHttp2Upgrade(Http2UpgradeHandler.java:158)
at io.undertow.server.protocol.http2.Http2UpgradeHandler.handleUpgradeBody(Http2UpgradeHandler.java:107)
at io.undertow.server.protocol.http2.Http2UpgradeHandler.handleRequest(Http2UpgradeHandler.java:97)
at io.undertow.server.handlers.DisallowedMethodsHandler.handleRequest(DisallowedMethodsHandler.java:61)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360)
at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
16:30:23,411 TRACE [io.undertow.server.HttpServerExchange] (default I/O-7) Starting to write response for HttpServerExchange{ OPTIONS * request {HTTP2-Settings=[AAEAABAAAAIAAAABAAQAAP//AAUAAEAA], Connection=[Upgrade, HTTP2-Settings], Upgrade=[h2c], User-Agent=[mod_cluster ping], Host=[127.0.0.1]} response {Connection=[keep-alive], Content-Length=[0], Date=[Wed, 23 Jan 2019 15:30:23 GMT]}}
{code}
> mod cluster and HTTP2 enabled (the default) not working
> -------------------------------------------------------
>
> Key: WFLY-11639
> URL: https://issues.jboss.org/browse/WFLY-11639
> Project: WildFly
> Issue Type: Feature Request
> Components: mod_cluster
> Affects Versions: 15.0.1.Final
> Environment: Mac OS X 10.14.3
> Tested on both
> AdoptOpenJDK 1.8.0_202-b08
> AdoptOpenJDK 11.0.1+13
> Reporter: Andreas Asplund
> Assignee: Radoslav Husar
> Priority: Major
>
> mod cluster and HTTP2 enabled (the default) when using listener default in the modcluster subsystem. When turning off HTTP2 the clustering starts working again. Nothing is printed in the logs by default so DEBUG logging has to be turned on for undertow.
> Steps to reproduce:
> {code}
> unzip wildfly-dist-15.0.1.Final.zip
> # Terminal 1
> cd wildfly-15.0.1.Final/bin/
> ./standalone.sh -c standalone-load-balancer.xml -Djava.net.preferIPv4Stack=true -b 127.0.0.1 -Djboss.modcluster.multicast.address=230.0.0.4
> # Terminal 2
> cd wildfly-15.0.1.Final/bin/
> ./standalone.sh -c standalone-ha.xml -Djboss.node.name=node1 -Djboss.socket.binding.port-offset=100 -Djboss.modcluster.multicast.address=230.0.0.4
> # Terminal 3
> cd wildfly-15.0.1.Final/bin/
> ./standalone.sh -c standalone-ha.xml -Djboss.node.name=node2 -Djboss.socket.binding.port-offset=200 -Djboss.modcluster.multicast.address=230.0.0.4
> # Terminal 4
> cd wildfly-15.0.1.Final/bin/
> ./jboss-cli.sh -c
> /subsystem=logging/logger=io.undertow:add(level=ALL)
> /subsystem=logging/console-handler=CONSOLE:write-attribute(name=level, value=ALL)
> connect localhost:10090
> /subsystem=logging/logger=io.undertow:add(level=ALL)
> /subsystem=logging/console-handler=CONSOLE:write-attribute(name=level, value=ALL)
> /subsystem=modcluster/proxy=default:write-attribute(name=listener,value=default)
> reload
> connect localhost:10190
> /subsystem=logging/logger=io.undertow:add(level=ALL)
> /subsystem=logging/console-handler=CONSOLE:write-attribute(name=level, value=ALL)
> /subsystem=modcluster/proxy=default:write-attribute(name=listener,value=default)
> reload
> {code}
> The following can be seen in the logs on the proxied servers:
> {code}
> 16:30:23,409 TRACE [io.undertow.request] (default I/O-7) Opened connection with /127.0.0.1:50002
> 16:30:23,410 DEBUG [io.undertow.request.error-response] (default I/O-7) Setting error code 500 for exchange HttpServerExchange{ OPTIONS * request {HTTP2-Settings=[AAEAABAAAAIAAAABAAQAAP//AAUAAEAA], Connection=[Upgrade, HTTP2-Settings], Upgrade=[h2c], User-Agent=[mod_cluster ping], Host=[127.0.0.1]} response {}}: java.lang.RuntimeException
> at io.undertow.server.HttpServerExchange.setStatusCode(HttpServerExchange.java:1410)
> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393)
> at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
> at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
> at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
> at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
> 16:30:23,410 DEBUG [io.undertow.request.io] (default I/O-7) UT005013: An IOException occurred: java.io.IOException: Invalid base64 character encountered: 47
> at io.undertow.util.FlexBase64$Decoder.nextByte(FlexBase64.java:1048)
> at io.undertow.util.FlexBase64$Decoder.nextByte(FlexBase64.java:1015)
> at io.undertow.util.FlexBase64$Decoder.decode(FlexBase64.java:1277)
> at io.undertow.util.FlexBase64$Decoder.decode(FlexBase64.java:1347)
> at io.undertow.util.FlexBase64$Decoder.decode(FlexBase64.java:1413)
> at io.undertow.util.FlexBase64$Decoder.access$500(FlexBase64.java:983)
> at io.undertow.util.FlexBase64.decodeURL(FlexBase64.java:320)
> at io.undertow.server.protocol.http2.Http2UpgradeHandler.handleHttp2Upgrade(Http2UpgradeHandler.java:158)
> at io.undertow.server.protocol.http2.Http2UpgradeHandler.handleUpgradeBody(Http2UpgradeHandler.java:107)
> at io.undertow.server.protocol.http2.Http2UpgradeHandler.handleRequest(Http2UpgradeHandler.java:97)
> at io.undertow.server.handlers.DisallowedMethodsHandler.handleRequest(DisallowedMethodsHandler.java:61)
> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360)
> at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
> at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
> at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
> at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
> 16:30:23,411 TRACE [io.undertow.server.HttpServerExchange] (default I/O-7) Starting to write response for HttpServerExchange{ OPTIONS * request {HTTP2-Settings=[AAEAABAAAAIAAAABAAQAAP//AAUAAEAA], Connection=[Upgrade, HTTP2-Settings], Upgrade=[h2c], User-Agent=[mod_cluster ping], Host=[127.0.0.1]} response {Connection=[keep-alive], Content-Length=[0], Date=[Wed, 23 Jan 2019 15:30:23 GMT]}}
> {code}
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
More information about the jboss-jira
mailing list