[JBoss JIRA] (JGRP-2355) TCP_NIO2 fails under Java 8
by Paul Ferraro (Jira)
Paul Ferraro created JGRP-2355:
----------------------------------
Summary: TCP_NIO2 fails under Java 8
Key: JGRP-2355
URL: https://issues.jboss.org/browse/JGRP-2355
Project: JGroups
Issue Type: Bug
Affects Versions: 4.1.1
Reporter: Paul Ferraro
Assignee: Bela Ban
Because the 4.1.x releases are built with JDK11, I see the following at runtime when running under Java 8:
{noformat}
WARN [org.jgroups.protocols.TCP_NIO2] (TQ-Bundler-6,ejb,node-1) node-1: failed sending message to 127.0.0.1:7700: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
{noformat}
The problematic code seems to be here:
https://github.com/belaban/JGroups/blob/master/src/org/jgroups/blocks/cs/...
In JDK8, Buffer.clear() was final and returned a Buffer object (hence the need for your code to cast). However, in JDK11 Buffer.clear() is no longer final, allowing subclasses to override the return type, which ByteBuffer indeed does (to return a ByteBuffer). However, since JGroups 4.1.x is built with JDK11, when running on Java 8, the method is not found.
There are 2 ways to fix this:
1. Ensure 4.1.x releases are built using JDK8 (since source is still compatible with Java 8)
2. Cast java.io.ByteBuffer to java.nio.Buffer when invoking clear() to avoid the NoSuchMethodError.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
6 years, 11 months
[JBoss JIRA] (WFLY-12214) JGRP000029: failed sending message: java.net.ConnectException: Connection refused
by Tommasso Borgato (Jira)
[ https://issues.jboss.org/browse/WFLY-12214?page=com.atlassian.jira.plugin... ]
Tommasso Borgato updated WFLY-12214:
------------------------------------
Description:
The error is observed in fail-over clustering tests where fail-over is "shutdown" and the jgroups stack is TCP.
The error was not observed in:
* wildfly-17.0.0.Final.zip
* wildfly-16.0.0.Final.zip
Right after one node (wildfly1) is shut down and restarted and the next node (wildfly2) is shut down we see:
{noformat}
2019-06-20 10:55:07,021 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN000094: Received new cluster view for channel ejb: [wildfly3|6] (3) [wildfly3, wildfly4, wildfly1]
2019-06-20 10:55:07,022 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN100001: Node wildfly2 left the cluster
2019-06-20 10:55:07,109 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
2019-06-20 10:55:07,251 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
2019-06-20 10:55:07,342 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
2019-06-20 10:55:07,437 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
{noformat}
Complete logs [here|https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/EAP7/vie...].
The number of errors is about 1000 per node;
Overall fail-rate is still close to 0%.
was:
The error is observed in fail-over clustering tests where fail-over is "shutdown" and the jgroups stack is TCP.
The error was not observed in:
wildfly-17.0.0.Final.zip
Right after one node (wildfly1) is shut down and restarted and the next node (wildfly2) is shut down we see:
{noformat}
2019-06-20 10:55:07,021 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN000094: Received new cluster view for channel ejb: [wildfly3|6] (3) [wildfly3, wildfly4, wildfly1]
2019-06-20 10:55:07,022 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN100001: Node wildfly2 left the cluster
2019-06-20 10:55:07,109 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
2019-06-20 10:55:07,251 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
2019-06-20 10:55:07,342 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
2019-06-20 10:55:07,437 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
{noformat}
Complete logs [here|https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/EAP7/vie...].
The number of errors is about 1000 per node;
Overall fail-rate is still close to 0%.
> JGRP000029: failed sending message: java.net.ConnectException: Connection refused
> ---------------------------------------------------------------------------------
>
> Key: WFLY-12214
> URL: https://issues.jboss.org/browse/WFLY-12214
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 18.0.0.Beta1
> Reporter: Tommasso Borgato
> Assignee: Paul Ferraro
> Priority: Major
>
> The error is observed in fail-over clustering tests where fail-over is "shutdown" and the jgroups stack is TCP.
> The error was not observed in:
> * wildfly-17.0.0.Final.zip
> * wildfly-16.0.0.Final.zip
> Right after one node (wildfly1) is shut down and restarted and the next node (wildfly2) is shut down we see:
> {noformat}
> 2019-06-20 10:55:07,021 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN000094: Received new cluster view for channel ejb: [wildfly3|6] (3) [wildfly3, wildfly4, wildfly1]
> 2019-06-20 10:55:07,022 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN100001: Node wildfly2 left the cluster
> 2019-06-20 10:55:07,109 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
> 2019-06-20 10:55:07,251 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
> 2019-06-20 10:55:07,342 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
> 2019-06-20 10:55:07,437 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
> {noformat}
> Complete logs [here|https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/EAP7/vie...].
> The number of errors is about 1000 per node;
> Overall fail-rate is still close to 0%.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
6 years, 11 months
[JBoss JIRA] (WFLY-12214) JGRP000029: failed sending message: java.net.ConnectException: Connection refused
by Tommasso Borgato (Jira)
[ https://issues.jboss.org/browse/WFLY-12214?page=com.atlassian.jira.plugin... ]
Tommasso Borgato updated WFLY-12214:
------------------------------------
Description:
The error is observed in fail-over clustering tests where fail-over is "shutdown" and the jgroups stack is TCP.
The error was not observed in:
wildfly-17.0.0.Final.zip
Right after one node (wildfly1) is shut down and restarted and the next node (wildfly2) is shut down we see:
{noformat}
2019-06-20 10:55:07,021 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN000094: Received new cluster view for channel ejb: [wildfly3|6] (3) [wildfly3, wildfly4, wildfly1]
2019-06-20 10:55:07,022 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN100001: Node wildfly2 left the cluster
2019-06-20 10:55:07,109 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
2019-06-20 10:55:07,251 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
2019-06-20 10:55:07,342 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
2019-06-20 10:55:07,437 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
{noformat}
Complete logs [here|https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/EAP7/vie...].
The number of errors is about 1000 per node;
Overall fail-rate is still close to 0%.
was:
The error is observed in fail-over clustering tests where fail-over is "shutdown" and the jgroups subsystem is as follows:
{noformat}
<subsystem xmlns="urn:jboss:domain:jgroups:7.0">
<channels default="ee">
<channel name="ee" stack="tcp" cluster="ejb"/>
</channels>
<stacks default="tcp">
<stack name="udp">
<transport type="UDP" socket-binding="jgroups-udp">
<property name="ip_ttl">
32
</property>
</transport>
<protocol type="PING"/>
<protocol type="MERGE3"/>
<socket-protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
<protocol type="FD_ALL"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="UFC"/>
<protocol type="MFC"/>
<protocol type="FRAG3"/>
</stack>
<stack name="tcp">
<transport type="TCP" socket-binding="jgroups-tcp"/>
<socket-protocol type="MPING" socket-binding="jgroups-mping"/>
<protocol type="MERGE3"/>
<socket-protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
<protocol type="FD_ALL"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="MFC"/>
<protocol type="FRAG3"/>
</stack>
</stacks>
</subsystem>
{noformat}
It wasn't observed with the previous version:
{noformat}
<subsystem xmlns="urn:jboss:domain:jgroups:6.0">
<channels default="ee">
<channel name="ee" stack="tcp" cluster="ejb"/>
</channels>
<stacks default="tcp">
<stack name="udp">
<transport type="UDP" socket-binding="jgroups-udp">
<property name="ip_ttl">
32
</property>
</transport>
<protocol type="PING"/>
<protocol type="MERGE3"/>
<protocol type="FD_SOCK"/>
<protocol type="FD_ALL"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="UFC"/>
<protocol type="MFC"/>
<protocol type="FRAG3"/>
</stack>
<stack name="tcp">
<transport type="TCP" socket-binding="jgroups-tcp"/>
<socket-protocol type="MPING" socket-binding="jgroups-mping"/>
<protocol type="MERGE3"/>
<protocol type="FD_SOCK"/>
<protocol type="FD_ALL"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="MFC"/>
<protocol type="FRAG3"/>
</stack>
</stacks>
</subsystem>
{noformat}
Right after one node (wildfly1) is shut down and restarted and the next node (wildfly2) is shut down we see:
{noformat}
2019-06-20 08:00:46,880 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 82) WFLYUT0021: Registered web context: '/clusterbench-granular' for server 'default-server'
2019-06-20 08:00:46,939 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 88) WFLYUT0021: Registered web context: '/clusterbench' for server 'default-server'
2019-06-20 08:00:47,024 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 85) WFLYUT0021: Registered web context: '/clusterbench-passivating' for server 'default-server'
2019-06-20 08:00:47,331 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0010: Deployed "clusterbench-ee8.ear" (runtime-name : "clusterbench-ee8.ear")
2019-06-20 08:00:47,335 INFO [org.jboss.as.server] (ServerService Thread Pool -- 47) WFLYSRV0010: Deployed "postgresql-connector.jar" (runtime-name : "postgresql-connector.jar")
2019-06-20 08:00:47,560 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
2019-06-20 08:00:47,562 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://10.0.146.117:9990/management
2019-06-20 08:00:47,563 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://10.0.146.117:9990
2019-06-20 08:00:47,563 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 18.0.0.Beta1-SNAPSHOT (WildFly Core 9.0.1.Final) started in 26227ms - Started 1065 of 1293 services (538 services are lazy, passive or on-demand)
2019-06-20 08:02:19,985 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-8,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (134 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:ejb, UNICAST3: DATA, seqno=12148, TP: [cluster=ejb]
2019-06-20 08:02:20,002 INFO [org.infinispan.CLUSTER] (thread-216,ejb,wildfly1) ISPN000094: Received new cluster view for channel ejb: [wildfly3|6] (3) [wildfly3, wildfly4, wildfly1]
2019-06-20 08:02:20,006 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-8,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (60 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: UNICAST3: ACK, seqno=284, conn_id=4, ts=109, TP: [cluster=ejb]
2019-06-20 08:02:20,027 INFO [org.infinispan.CLUSTER] (thread-216,ejb,wildfly1) ISPN100001: Node wildfly2 left the cluster
2019-06-20 08:02:20,031 INFO [org.infinispan.CLUSTER] (thread-216,ejb,wildfly1) ISPN000094: Received new cluster view for channel ejb: [wildfly3|6] (3) [wildfly3, wildfly4, wildfly1]
{noformat}
Whole logs [here|https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/EAP7/vie...].
The number of errors is about 3000 per node;
Overall fail-rate is still low: about 0.55%, but it has increased sensibly if compared to the previous version where it was about 0.01%.
> JGRP000029: failed sending message: java.net.ConnectException: Connection refused
> ---------------------------------------------------------------------------------
>
> Key: WFLY-12214
> URL: https://issues.jboss.org/browse/WFLY-12214
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 18.0.0.Beta1
> Reporter: Tommasso Borgato
> Assignee: Paul Ferraro
> Priority: Major
>
> The error is observed in fail-over clustering tests where fail-over is "shutdown" and the jgroups stack is TCP.
> The error was not observed in:
> wildfly-17.0.0.Final.zip
> Right after one node (wildfly1) is shut down and restarted and the next node (wildfly2) is shut down we see:
> {noformat}
> 2019-06-20 10:55:07,021 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN000094: Received new cluster view for channel ejb: [wildfly3|6] (3) [wildfly3, wildfly4, wildfly1]
> 2019-06-20 10:55:07,022 INFO [org.infinispan.CLUSTER] (thread-84,ejb,wildfly1) ISPN100001: Node wildfly2 left the cluster
> 2019-06-20 10:55:07,109 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
> 2019-06-20 10:55:07,251 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
> 2019-06-20 10:55:07,342 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
> 2019-06-20 10:55:07,437 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ejb,wildfly1) JGRP000029: wildfly1: failed sending message to wildfly2 (133 bytes): java.net.ConnectException: Connection refused (Connection refused), headers: FORK: ejb:web, UNICAST3: DATA, seqno=44773, TP: [cluster=ejb]
> {noformat}
> Complete logs [here|https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/EAP7/vie...].
> The number of errors is about 1000 per node;
> Overall fail-rate is still close to 0%.
--
This message was sent by Atlassian Jira
(v7.12.1#712002)
6 years, 11 months