11:02 a.m.
Reduce size of large headers: place data into message itself
------------------------------------------------------------
Key: JGRP-1259
URL: https://jira.jboss.org/browse/JGRP-1259
Project: JGroups
Issue Type: Feature Request
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 2.12
When we send a view around, we essentially create an empty Message and attach a
GmsHeader.VIEW with the new view.
If the view consists of 200 members, the view header will be very big and the message is
still empty (null buffer).
The issue here is that FRAG2 only fragments the message's buffer (this is faster), and
so a huge header might add so much to the size of a serialized message that the transport
(UDP) won't be able to send it !
SOLUTION: still use the header (e.g. GmsHeader.VIEW), but place potentially large payloads
into the message. This will allow FRAG2 to fragment the message very efficiently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:19 a.m.
New subject: [JBoss JIRA] Commented: (JGRP-1259) Reduce size of large headers: place data into message itself
[
https://issues.jboss.org/browse/JGRP-1259?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1259:
--------------------------------
The problem is that FRAG2 is at the top of the stack (needs to be there because it has to
be ovre flow control somewhere) Subsequent protocols further down add headers, so FRAG2
should be just over the transport, but that canot be done because of flow control... we
*could* have a FRAG2 at the top *and* a FRAG protocol (which marshalls the entire message
including headers), but this is overhead...
Reduce size of large headers: place data into message itself
------------------------------------------------------------
Key: JGRP-1259
URL: https://issues.jboss.org/browse/JGRP-1259
Project: JGroups
Issue Type: Feature Request
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 2.12
When we send a view around, we essentially create an empty Message and attach a
GmsHeader.VIEW with the new view.
If the view consists of 200 members, the view header will be very big and the message is
still empty (null buffer).
The issue here is that FRAG2 only fragments the message's buffer (this is faster),
and so a huge header might add so much to the size of a serialized message that the
transport (UDP) won't be able to send it !
SOLUTION: still use the header (e.g. GmsHeader.VIEW), but place potentially large
payloads into the message. This will allow FRAG2 to fragment the message very efficiently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:21 a.m.
New subject: [JBoss JIRA] Commented: (JGRP-1259) Reduce size of large headers: place data into message itself
[
https://issues.jboss.org/browse/JGRP-1259?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1259:
--------------------------------
Alternative: create a new protocol (HEADER_COMPRESSION) which is layered over transport
and (a second) FRAG2 (which is more efficient than FRAG): UDP:FRAG2(id=2322):HEADER_COMP.
HEADER_COMP would check if the payload is null, and - if so - marshall the headers
directly into the payload.
Not good because the payload is not always null, so we'd still need FRAG... :-(
Reduce size of large headers: place data into message itself
------------------------------------------------------------
Key: JGRP-1259
URL: https://issues.jboss.org/browse/JGRP-1259
Project: JGroups
Issue Type: Feature Request
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 2.12
When we send a view around, we essentially create an empty Message and attach a
GmsHeader.VIEW with the new view.
If the view consists of 200 members, the view header will be very big and the message is
still empty (null buffer).
The issue here is that FRAG2 only fragments the message's buffer (this is faster),
and so a huge header might add so much to the size of a serialized message that the
transport (UDP) won't be able to send it !
SOLUTION: still use the header (e.g. GmsHeader.VIEW), but place potentially large
payloads into the message. This will allow FRAG2 to fragment the message very efficiently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:21 a.m.
New subject: [JBoss JIRA] Commented: (JGRP-1259) Reduce size of large headers: place data into message itself
[
https://issues.jboss.org/browse/JGRP-1259?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1259:
--------------------------------
The biggest header is always one which contains a View, or a list of cluster members.
Typically this is a View, so if we compress View, then we would not really need this JIRA
for large clusters...
Reduce size of large headers: place data into message itself
------------------------------------------------------------
Key: JGRP-1259
URL: https://issues.jboss.org/browse/JGRP-1259
Project: JGroups
Issue Type: Feature Request
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 2.12
When we send a view around, we essentially create an empty Message and attach a
GmsHeader.VIEW with the new view.
If the view consists of 200 members, the view header will be very big and the message is
still empty (null buffer).
The issue here is that FRAG2 only fragments the message's buffer (this is faster),
and so a huge header might add so much to the size of a serialized message that the
transport (UDP) won't be able to send it !
SOLUTION: still use the header (e.g. GmsHeader.VIEW), but place potentially large
payloads into the message. This will allow FRAG2 to fragment the message very efficiently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:27 a.m.
New subject: [JBoss JIRA] Commented: (JGRP-1259) Reduce size of large headers: place data into message itself
[
https://issues.jboss.org/browse/JGRP-1259?page=com.atlassian.jira.plugin....
]
Bela Ban commented on JGRP-1259:
--------------------------------
Hmm, View cannot esaily be compressed. Sending only the ViewId and a list of indices (one
for each member) would only work if everybody had the ViewId available. This would work
for V1={A,B,C,D} --> V2={A,B,C} (D leaving), but not for V1={A,B,C} --> V2={A,B,C,D}
(D joining)... unless we serialized D explicitly here... This needs more thought. A
similar effect could be had by canonicalizing Addresses into IDs, but for different
reasons this was pushed into 3.x
Reduce size of large headers: place data into message itself
------------------------------------------------------------
Key: JGRP-1259
URL: https://issues.jboss.org/browse/JGRP-1259
Project: JGroups
Issue Type: Feature Request
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 2.12
When we send a view around, we essentially create an empty Message and attach a
GmsHeader.VIEW with the new view.
If the view consists of 200 members, the view header will be very big and the message is
still empty (null buffer).
The issue here is that FRAG2 only fragments the message's buffer (this is faster),
and so a huge header might add so much to the size of a serialized message that the
transport (UDP) won't be able to send it !
SOLUTION: still use the header (e.g. GmsHeader.VIEW), but place potentially large
payloads into the message. This will allow FRAG2 to fragment the message very efficiently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:34 a.m.
New subject: [JBoss JIRA] Resolved: (JGRP-1259) Reduce size of large headers: place data into message itself
[
https://issues.jboss.org/browse/JGRP-1259?page=com.atlassian.jira.plugin....
]
Bela Ban resolved JGRP-1259.
----------------------------
Resolution: Deferred
Won't do this for the reasons outlined in the comments. However, this is a good
practice for some selected protocols. E.g. RELAY uses this. But it has to be the only
protocol that's serialing into the payload, or else different protocols would
overwrite each other's data.
Reduce size of large headers: place data into message itself
------------------------------------------------------------
Key: JGRP-1259
URL: https://issues.jboss.org/browse/JGRP-1259
Project: JGroups
Issue Type: Feature Request
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 2.12
When we send a view around, we essentially create an empty Message and attach a
GmsHeader.VIEW with the new view.
If the view consists of 200 members, the view header will be very big and the message is
still empty (null buffer).
The issue here is that FRAG2 only fragments the message's buffer (this is faster),
and so a huge header might add so much to the size of a serialized message that the
transport (UDP) won't be able to send it !
SOLUTION: still use the header (e.g. GmsHeader.VIEW), but place potentially large
payloads into the message. This will allow FRAG2 to fragment the message very efficiently.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
5082
days inactive
5134
days old
5 comments
1 participants
participants (1)
-
Bela Ban (JIRA)