InputStream is processed after the different version error ---------------------------------------------------------- Key: JGRP-2298 URL: https://issues.jboss.org/browse/JGRP-2298 Project: JGroups Issue Type: Bug Affects Versions: 4.0.11, 4.0.13 Environment: OS: Debian 8 Reporter: Boris Kantor Assignee: Bela Ban Fix For: 4.0.16 Occasionally JGroups shows up a warning when receiving a message with a different JGroups version: {{JGRP000010: packet from 10.10.10.130:7800 has different version (0.0.0) than ours (4.0.11); packet is discarded | (Log4J2LogImpl.java:91)}} Although the message is then discarded, the socket connection will be kept open. The input stream still contains data that will be processed and falsly interpreted as the next message. If you have bad luck, the next bytes will match the correct version number. This leads then to following errors: {code:java} 2018-07-17 10:15:34.052 [Connect | ] WARN TCP - JGRP000010: packet from 10.10.10.130:7800 has different version (0.0.0) than ours (4.0.11); packet is discarded | (Log4J2LogImpl.java:91) 2018-07-17 10:15:34.059 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.lang.ClassNotFoundException: Class for magic number 8237 cannot be found at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:118) at org.jgroups.Message.readHeader(Message.java:869) at org.jgroups.Message.readFrom(Message.java:742) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.065 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.lang.ClassNotFoundException: Class for magic number 8237 cannot be found at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:118) at org.jgroups.Message.readHeader(Message.java:869) at org.jgroups.Message.readFrom(Message.java:742) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.072 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.io.IOException: length has to be 4 or 16 bytes (was 47 bytes) at org.jgroups.stack.IpAddress.readFrom(IpAddress.java:171) at org.jgroups.util.Util.readAddress(Util.java:1379) at org.jgroups.Message.readFrom(Message.java:731) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.076 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.lang.ClassNotFoundException: Class for magic number 28789 cannot be found at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:118) at org.jgroups.Message.readHeader(Message.java:869) at org.jgroups.Message.readFrom(Message.java:742) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.077 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.lang.ClassNotFoundException: Class for magic number 8237 cannot be found at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:118) at org.jgroups.Message.readHeader(Message.java:869) at org.jgroups.Message.readFrom(Message.java:742) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.078 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.io.IOException: length has to be 4 or 16 bytes (was 47 bytes) at org.jgroups.stack.IpAddress.readFrom(IpAddress.java:171) at org.jgroups.util.Util.readAddress(Util.java:1379) at org.jgroups.Message.readFrom(Message.java:731) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.078 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.lang.ClassNotFoundException: Class for magic number 8237 cannot be found at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:118) at org.jgroups.Message.readHeader(Message.java:869) at org.jgroups.Message.readFrom(Message.java:742) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.079 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.io.IOException: length has to be 4 or 16 bytes (was 47 bytes) at org.jgroups.stack.IpAddress.readFrom(IpAddress.java:171) at org.jgroups.util.Util.readAddress(Util.java:1379) at org.jgroups.Message.readFrom(Message.java:731) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.080 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.lang.ClassNotFoundException: Class for magic number 12320 cannot be found at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:118) at org.jgroups.Message.readHeader(Message.java:869) at org.jgroups.Message.readFrom(Message.java:742) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.089 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.lang.ClassNotFoundException: Class for magic number 25193 cannot be found at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:118) at org.jgroups.Message.readHeader(Message.java:869) at org.jgroups.Message.readFrom(Message.java:742) at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1346) at org.jgroups.protocols.TP.receive(TP.java:1323) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) 2018-07-17 10:15:34.107 [Connect | ] ERROR TCP - JGRP000030: jgm113: failed handling incoming message | (Log4J2LogImpl.java:116) java.lang.ClassNotFoundException: Class for magic number 25970 cannot be found at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:118) at org.jgroups.Message.readHeader(Message.java:869) at org.jgroups.Message.readFrom(Message.java:742) at org.jgroups.util.Util.readMessageBatch(Util.java:1193) at org.jgroups.protocols.TP.handleMessageBatch(TP.java:1329) at org.jgroups.protocols.TP.receive(TP.java:1321) at org.jgroups.blocks.cs.BaseServer.receive(BaseServer.java:171) at org.jgroups.blocks.cs.TcpConnection$Receiver.run(TcpConnection.java:290) at eu.application.nac.cluster.jgroups.JGroupsTask.execute(JGroupsTask.java:41) at eu.application.nac.engine.scheduling.Task.run(Task.java:339) at java.lang.Thread.run(Thread.java:745) {code} When it comes to the payload of the message the size of the payload can be as big as 2GB ({{len=in.readInt();}}). On small machines this will lead to an OutOfMemoryError. There is a need for some handling of such misinterpreted messages. Like closing and reopening a socket connection or completely ignoring the following bytes until a new message begins or an other algorithm that would prevent the processing of the remaining bytes.