[jboss-jira] [JBoss JIRA] (JGRP-2494) DNS_PING and failed probes

Wed Jul 22 14:01:21 EDT 2020

     [ https://issues.redhat.com/browse/JGRP-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dewayne McNair updated JGRP-2494:
---------------------------------
    Steps to Reproduce: 
logs are below showing the DNS_PING query where the two entries are found - port 8888 is the web port, port 8889 is the jgroups port.  Undertow logs a failure for the probe at DEBUG level, but jgroups is logging the failed probe at ERROR level.

{code:java}
2020-07-22 17:51:03,356 DEBUG [org.jgroups.protocols.dns.DNS_PING] (thread-5,null,463dfca1ff73) 463dfca1ff73: entries collected from DNS (in 3 ms): [172.29.11.55:8889, 172.29.11.55:8888]2020-07-22 17:51:03,356 DEBUG [org.jgroups.protocols.dns.DNS_PING] (thread-5,null,463dfca1ff73) 463dfca1ff73: entries collected from DNS (in 3 ms): [172.29.11.55:8889, 172.29.11.55:8888]2020-07-22 17:51:03,356 DEBUG [org.jgroups.protocols.dns.DNS_PING] (thread-5,null,463dfca1ff73) 463dfca1ff73: sending discovery requests to [172.29.11.55:8889, 172.29.11.55:8888]2020-07-22 17:51:03,357 DEBUG [io.undertow.request.io] (default I/O-2) UT005014: Failed to parse request: io.undertow.util.BadRequestException: UT000165: Invalid character  in request-target at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpRequestParser.handlePath(HttpRequestParser.java:389) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpRequestParser.handle(HttpRequestParser.java:248) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:187) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:162) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:100) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:57) at org.jboss.xnio at 3.7.7.Final//org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.jboss.xnio at 3.7.7.Final//org.xnio.ChannelListeners$10.handleEvent(ChannelListeners.java:291) at org.jboss.xnio at 3.7.7.Final//org.xnio.ChannelListeners$10.handleEvent(ChannelListeners.java:286) at org.jboss.xnio at 3.7.7.Final//org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.jboss.xnio.nio at 3.7.7.Final//org.xnio.nio.QueuedNioTcpServer2.acceptTask(QueuedNioTcpServer2.java:178) at org.jboss.xnio.nio at 3.7.7.Final//org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:612) at org.jboss.xnio.nio at 3.7.7.Final//org.xnio.nio.WorkerThread.run(WorkerThread.java:479) 2020-07-22 17:51:03,358 ERROR [org.jgroups.protocols.TCP] (Connection.Receiver [172.17.0.10:57508 - 172.29.11.55:8888]-22,ejb,463dfca1ff73) JGRP000030: 463dfca1ff73: failed handling incoming message: java.io.EOFException at java.base/java.io.DataInputStream.readFully(DataInputStream.java:202) at org.jgroups at 4.1.9.Final//org.jgroups.util.Util.readMessageBatch(Util.java:1226) at org.jgroups at 4.1.9.Final//org.jgroups.protocols.TP.handleMessageBatch(TP.java:1448) at org.jgroups at 4.1.9.Final//org.jgroups.protocols.TP.receive(TP.java:1440) at org.jgroups at 4.1.9.Final//org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups at 4.1.9.Final//org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:300) at org.jboss.as.clustering.common at 19.1.0.Final//org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49) at org.jboss.as.clustering.common at 19.1.0.Final//org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70) at java.base/java.lang.Thread.run(Thread.java:834)
{code}

  was:
logs are below showing the DNS_PING query where the two entries are found - port 8888 is the web port, port 8889 is the jgroups port.  Undertow logs a failure for the probe at DEBUG level, but jgroups is logging the failed probe at ERROR level.

{noformat}
2020-07-22 17:51:03,356 DEBUG [org.jgroups.protocols.dns.DNS_PING] (thread-5,null,463dfca1ff73) 463dfca1ff73: entries collected from DNS (in 3 ms): [172.29.11.55:8889, 172.29.11.55:8888]2020-07-22 17:51:03,356 DEBUG [org.jgroups.protocols.dns.DNS_PING] (thread-5,null,463dfca1ff73) 463dfca1ff73: entries collected from DNS (in 3 ms): [172.29.11.55:8889, 172.29.11.55:8888]2020-07-22 17:51:03,356 DEBUG [org.jgroups.protocols.dns.DNS_PING] (thread-5,null,463dfca1ff73) 463dfca1ff73: sending discovery requests to [172.29.11.55:8889, 172.29.11.55:8888]2020-07-22 17:51:03,357 DEBUG [io.undertow.request.io] (default I/O-2) UT005014: Failed to parse request: io.undertow.util.BadRequestException: UT000165: Invalid character  in request-target at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpRequestParser.handlePath(HttpRequestParser.java:389) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpRequestParser.handle(HttpRequestParser.java:248) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:187) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:162) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:100) at io.undertow.core at 2.1.0.Final//io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:57) at org.jboss.xnio at 3.7.7.Final//org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.jboss.xnio at 3.7.7.Final//org.xnio.ChannelListeners$10.handleEvent(ChannelListeners.java:291) at org.jboss.xnio at 3.7.7.Final//org.xnio.ChannelListeners$10.handleEvent(ChannelListeners.java:286) at org.jboss.xnio at 3.7.7.Final//org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) at org.jboss.xnio.nio at 3.7.7.Final//org.xnio.nio.QueuedNioTcpServer2.acceptTask(QueuedNioTcpServer2.java:178) at org.jboss.xnio.nio at 3.7.7.Final//org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:612) at org.jboss.xnio.nio at 3.7.7.Final//org.xnio.nio.WorkerThread.run(WorkerThread.java:479) 2020-07-22 17:51:03,358 ERROR [org.jgroups.protocols.TCP] (Connection.Receiver [172.17.0.10:57508 - 172.29.11.55:8888]-22,ejb,463dfca1ff73) JGRP000030: 463dfca1ff73: failed handling incoming message: java.io.EOFException at java.base/java.io.DataInputStream.readFully(DataInputStream.java:202) at org.jgroups at 4.1.9.Final//org.jgroups.util.Util.readMessageBatch(Util.java:1226) at org.jgroups at 4.1.9.Final//org.jgroups.protocols.TP.handleMessageBatch(TP.java:1448) at org.jgroups at 4.1.9.Final//org.jgroups.protocols.TP.receive(TP.java:1440) at org.jgroups at 4.1.9.Final//org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups at 4.1.9.Final//org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:300) at org.jboss.as.clustering.common at 19.1.0.Final//org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49) at org.jboss.as.clustering.common at 19.1.0.Final//org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70) at java.base/java.lang.Thread.run(Thread.java:834)
{noformat}

> DNS_PING and failed probes
> --------------------------
>
>                 Key: JGRP-2494
>                 URL: https://issues.redhat.com/browse/JGRP-2494
>             Project: JGroups
>          Issue Type: Bug
>    Affects Versions: 4.1.9
>            Reporter: Dewayne McNair
>            Assignee: Bela Ban
>            Priority: Major
>
> Using DNS_PING in a DC/OS (marathon) deployment, there are two SRV records returned - one for the web port and one for the jgroups port.  When the discovery request is sent, an exception is thrown and logged.  The clustering is working, but logs are filling up due to the exception happening and being logged frequently.

--
This message was sent by Atlassian Jira
(v7.13.8#713008)