[jboss-jira] [JBoss JIRA] Commented: (JGRP-1331) Digest: remove low seqno and compress highest_delivered and highest_received seqnos

[Bela Ban (JIRA)]

    [ https://issues.jboss.org/browse/JGRP-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12625732#comment-12625732 ] 

Bela Ban commented on JGRP-1331:
--------------------------------

OK, so the following optimizations were done:

- low seqno was removed from the digest

- highest_delivered and highest_received were compressed (Util.encodeLongSequence()). Since we know that highest_received is usually only a small offset to highest_delivered, we can only send the delta

- Added a generic method Util.encodeLong(long num) to send only the relevant bits of a long across the wire. E.g. a seqno=30000 uses only 2 bytes (plus 1 length byte), instead of 8 !

- Applied Util.encodeLong() and Util.encodeLongSequence() to a bunch of classes, e.g. Range, NakAckHeader.seqno etc. Whenever we send longs that have only a few bits set, this compression reduces the data sent over the wire dramatically !

- Reduced the serialized JoinRsp by sending the member list only once, and not twice (in the view and the digest). A JoinRsp containing a 1000 members generates 43'000 bytes of serialized data, but onyl 23'000 bytes of serialized data with the new compressed code. See SizeTest.testlargeJoinRsp()


> Digest: remove low seqno and compress highest_delivered and highest_received seqnos
> -----------------------------------------------------------------------------------
>
>                 Key: JGRP-1331
>                 URL: https://issues.jboss.org/browse/JGRP-1331
>             Project: JGroups
>          Issue Type: Task
>            Reporter: Bela Ban
>            Assignee: Bela Ban
>             Fix For: 3.0
>
>
> We don't need Digest.Entry.low_seqno for NakReceiverWindow anymore. We only need it for the Retransmitter implementations. If we can remove that last dependency, e.g. by storing the last seqno removed from a retransmitter, we can remove low_seqno from a digest. This is significant, for example if we have 100 nodes, a digest includes 300 longs:(low+highest_received+highest_delivered) * 100. If we remove low_seqno, we only have 200 longs.
> Also, if we could compress the highest_delivered and highest_received seqnos, we could save even more space to send digests across the network.
> This is important for large clusters, to reduce the size of STABLE messages and views.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira