[ http://jira.jboss.com/jira/browse/JGRP-324?page=all ] Bela Ban updated JGRP-324: -------------------------- Assignee: Richard Achmatowicz (was: Bela Ban) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.jboss.com/jira/browse/JGRP-324?page=comments#action_12417636 ] Richard Achmatowicz commented on JGRP-324: ------------------------------------------ Adding in some comments which were previously sent by email: Wireformats for UDP JGroups peers can receive UDP packets, where the destination address may be a unicast address or a multicast address. Both types of packets have the same payload format (i.e. when we strip off the UDP header, we are left with a payload with the same format). It is this payload which Wireshark will have to dissect. JGroups uses message bundling to put several Messages into one UDP packet. Bundling can either be turned on or turned off, so we have to be able to handle both formats. UDP payload (non-bundled) ------------------------- version (2 bytes) flags (1 byte) Message (variable length) UDP payload (bundled) --------------------- version (2 bytes) flags (1 byte) length (4 bytes) Address (variable length) Message (first bundled message - variable length) Message (second bundled message variable length) ... Message ('length' bundled message- variable length) NOTES: 1. In order to dissect these packets, we need to be able to dissect Address and Message. Address is read using Util.readAddress(). Message is read using Message.readFrom() 2. version format: e.g. JGroups 2.6.2 * X = 0-31 for major versions * Y = 0-31 for minor versions * Z = 0-63 for micro versions See the org.jgroups.Version class for algorithm on how to decode the version. 3. flags format: LIST (0x1) MULITCAST (0x10) OOB (0x100) 4. We can tell whether bundling is turned on by the value the expression (flags & LIST): * if (flags & LIST == false), then after reading version and flags, we expect only one message in the packet * if (flags & LIST == true), then after reading version and flags, we read length and expect 'length' messages in the packet In order to dissect the UDP payload, we need to be able to dissect serialized Address and Message objects. ------------------ Address wireformat ------------------ Address is a JGroups class which represents an 'address'. Depending on the transport we are using, this could be an <IP address, port> combination, or a JMS topic. The in-memory value of an Address is written/serialized using Util.writeAddress(). The wireformat is: isnull (variable length) isIpAddress (variable length) contents (variable length) The values isnull ("is the address equal to null?") and isIpAddress("is the address an IpAddress instance") are encoded Boolean values. Contents is variable and involves shipping classes across the wire. See the method org.jgroups.util.Util.writeAddress() for the specifc details. ------------------ Message wireformat ------------------ Message is a JGroups class and represents data shipped around the system. Messages contain the following data elements: dest_addr - destination address of the Message src_addr - source address of the Message flags - flags data - payload of the message list of headers - headers which Protocols use to store information Messages are written to the wire using the method org.jgroups.Message.writeTo(). The wireformat is: leading (1 byte) flags (1 byte) dest_addr (variable length) src_addr (variable length) buflen (4 bytes) buf ('buflen' bytes) numhdrs (2 bytes) (Key, Header) (first Header - variable length) (Key, Header) (second Header -variable length) ... (Key, Header) ('numhdrs' Header - variable length) NOTES: 1. leading is a set of flags indicating which elements have been written to the wireformat and which have not. DEST_SET (0x0000001) SRC_SET (0x0000010) BUF_SET (0x0000100) IPADDR_DEST (0x0010000) IPADDR_SRC (0x0100000) SRC_HOST_NULL (0x1000000) 2. flags indicate properties of the Message, whether it is out of bound, etc. OOB (0x001) LOW_PRIO (0x010) HIGH_PRIO (0x100) 3. dest_addr and src_addr are written using either IpAddress.writeTo() or Util.writeAddress(), depending on whether the Address is an IpAddress instance or not. I suspect that almost all will be IpAddress instances. See the method org.jgroups.Message.writeTo() for the specifc details on how Message is serialized. ------------------------ (Key, Header) wireformat ------------------------ Header is a JGroups class, and each protocol will have its own versions (e.g. TPHeader, FRAGHeader, etc.) which are subclasses of Header. Each Message has zero or more Headers associated with it. Each Header entry consists of a key and a value (e.g. ("UDP", TPHeader), ("FRAG", FRAGHeader), etc.) The wireformat is: key (variable length) magic (2 bytes) class (variable length, only if magic == -1) size (2 bytes) contents (variable length) NOTES: 1. key is the UTF8 encoding of the key String (e.g. "FRAG") 2. magic is the magic number of the Header class 3. class is the UTF8 encidong of the Header class name, and is only present if magic == -1 4. size is the size of the serialized Header class 5. contents is the serialized Header class contents, which is created by: Header class is streamable -> Header.writeTo() Header class is non-streamable -> Header.writeExternal() Comments -------- 1. We need to dissect and represent Addresses 2. We need to dissect and represent Messages and their contents. 3. We need to dissect and represent Headers and their contents. Given that Headers are classes of varying types, we just need a representation for the key attributes of the class. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.jboss.com/jira/browse/JGRP-324?page=all ] Richard Achmatowicz updated JGRP-324: ------------------------------------- Summary: Wireshark plugin for JGroups (was: Ethereal plugin for JGroups) Description: Write a C implementation of a Wireshark plugin which understands the JGroups wire format and displays information about the JGroups messages (e.g. sender, receiver, headers) in ethereal. (was: Write a C implementation of an ethereal plugin which understands the JGroups wire format and displays information about the JGroups messages (e.g. sender, receiver, headers) in ethereal.) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.jboss.com/jira/browse/JGRP-324?page=comments#action_12419806 ] Richard Achmatowicz commented on JGRP-324: ------------------------------------------ I've been investigating modifying the plugin to handle TCP based JGroups traffic, and it will be very difficult to achieve without a small modification to the JGroups wireformat. Here's the problem. When dissecting JGroups over TCP, we have to take account of two things due to the stream-orientation of TCP: (i) a large TP bundle may be split into several TCP fragments (ii) several small TP bundles may be placed into one TCP fragment Reassembly of TP bundles will occur at the receiving TCP peer end, but Wireshark captures the fragments before reassembly, in between the peers so to speak, and so the Wireshark plugin writer has to do the reassembly herself. Reassembling TCP fragments is very very simple if we have a field in the TP header which describes the length of the entire TP bundle. We can then say to Wireshark - "reassmble enough TCP fragments until you get a buffer with at least this length". Dissection on a complete TP bundle can then proceeed in a straightforward fashion. The TP header does not have a "TP bundle length" field at the moment. Without this information, we never know if we have a complete TP bundle in the buffer, and so have to dissect the TCP fragment "on the fly" - checking before each read from the buffer if we have enough data in the buffer left; in that case that there is not enough data, request the reading of another TCP fragment. I've had a go at implementing this, but it is just too complcated. It's easy to check before each read from the buffer if there is enough data available. It's also easy to get Wireshark to read another TCP fragment into the buffer. The real complication is in returning to the point where processing left off and contuniing, as the dissector is called anew each time we add in a new TCP fragment to the buffer. It is possible, but it would make the dissector extremely complicated and difficult to maintain. At present, the disseector can handle UDP packets (as far as I am aware) for 2.6.2, which is what I have been testing with, and any wire compatible format. If we could add a TP bundle length field to 2.7, this would allow handling both UDP and TCP for 2.7 onwards. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JGRP-324?page=com.atlassian.jira.plugi... ] Bela Ban commented on JGRP-324: ------------------------------- Richard, it would be good if we can tie the type of dissector to the version of JGroups. This should be simple IMO, as the version ID (the short) is the first thing a JGroups message contains. I have a feeling the wire format's going to change once more in 2.8, with logical addresses, so it would be good if we can already now tie the dissector to the version, so when/if the wire format changes, we can still parse messages from previous versions. Do you think this is doable ? -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JGRP-324?page=com.atlassian.jira.plugi... ] Richard Achmatowicz commented on JGRP-324: ------------------------------------------ Added in user profile options for controlling whether or not messages and headers are collected under subtrees: Display a Messages subtree to collect mesages [] Display a Headers subtree to collect headers [] Setting these options to false removes the Messages and Headers subtrees and avoids some of the extra drilldowns which were required to get to Message and Header information. The new UI is pictured in the attachment Wireshark-subtree-options.jpg -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://jira.jboss.org/jira/browse/JGRP-324?page=com.atlassian.jira.plugi... ] Richard Achmatowicz commented on JGRP-324: ------------------------------------------ Released a new version (0.1.0) of the dissector, to replace the draft version. Here are the main changes from the change log: 7 November 2008: (version 0.1.0) - Moved from port-based dissection (where the JGroups dissector is only applied if the source or dest port of the packet packet matches a specified port) to heuristic dissection (where the JGroups dissector is applied if the payload of the packet is identified as a JGroups message). - Added in support for checking wireformat version and whether or not the plugin supports that wireformat version. At present, only 2.6.4 and binary-compatible versions are supported. - Added in user profile settings to turn on/turn off display of Message subtrees and Header subtrees in UI. This reduces the number of drill downs required to get at the data in the captured packets. One way to move this forward will be to complete the basic list of headers to be processed. These are the headers which have already been implemented: * TP/TCP/UDP * PING * NAKACK * UNICAST * GMS * FDSOCK These are some of the headers we need to go in (according to Bela and Vladimir): * FD / FD_ALL * FC * STABLE * STATE_TRANSFER * MERGE2 -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira