[JBoss JIRA] (JGRP-2218) New payload interface
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Bela Ban edited comment on JGRP-2218 at 9/13/17 10:11 AM:
----------------------------------------------------------
Re {{size()}}: this is just an estimation to create a buffer that can hold the serialized data, without having to expand. So, you can always give an estimate that's higher than the actual serialized data. If the estimate is smaller, the buffer will expand, although at the cost of an additional byte[] array allocation. But since we're reusing the same output buffer, this will almost never happen anyway. So forget {{size()}}; it is not really relevant here... :-)
The size of a string can be computed efficiently, I have methods in Util which do that.
Also note that you don't have to use {{ObjectPayload}}, might as well use {{ByteArrayPayload}} and serialize, as you do now.
was (Author: belaban):
Re {{size()}}: this is just an estimation to create a buffer that can hold the serialized data, without having to expand. So, you can always give an estimate that's higher than the actual serialized data. If the estimate is smaller, the buffer will expand, although at the cost of an additional byte[] array allocation. But since we're reusing the same output buffer, this will almost never happen anyway. So forget {{size()}}; it is not really relevant here... :-)
The size of a string can be computed efficiently, I have methods in Util which do that.
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectBuffer}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload management
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadManagement}} (or {{PayloadPool}} component for a new buffer. A naive implementation might create a new buffer every time, and more sophisticated one might use a pool of payloads.
> The {{PayloadManagement}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 9 months
[JBoss JIRA] (JGRP-2218) New payload interface
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2218:
--------------------------------
Re {{size()}}: this is just an estimation to create a buffer that can hold the serialized data, without having to expand. So, you can always give an estimate that's higher than the actual serialized data. If the estimate is smaller, the buffer will expand, although at the cost of an additional byte[] array allocation. But since we're reusing the same output buffer, this will almost never happen anyway. So forget {{size()}}; it is not really relevant here... :-)
The size of a string can be computed efficiently, I have methods in Util which do that.
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectBuffer}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload management
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadManagement}} (or {{PayloadPool}} component for a new buffer. A naive implementation might create a new buffer every time, and more sophisticated one might use a pool of payloads.
> The {{PayloadManagement}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 9 months
[JBoss JIRA] (WFCORE-3286) It is possible to add multiple http-connectors to the same undertow listener
by Jiri Ondrusek (JIRA)
[ https://issues.jboss.org/browse/WFCORE-3286?page=com.atlassian.jira.plugi... ]
Jiri Ondrusek moved JBEAP-13147 to WFCORE-3286:
-----------------------------------------------
Project: WildFly Core (was: JBoss Enterprise Application Platform)
Key: WFCORE-3286 (was: JBEAP-13147)
Workflow: GIT Pull Request workflow (was: CDW with loose statuses v1)
Component/s: Remoting
(was: Remoting)
(was: Web (Undertow))
Affects Version/s: 3.0.1.Final
(was: 7.1.0.DR16)
> It is possible to add multiple http-connectors to the same undertow listener
> ----------------------------------------------------------------------------
>
> Key: WFCORE-3286
> URL: https://issues.jboss.org/browse/WFCORE-3286
> Project: WildFly Core
> Issue Type: Bug
> Components: Remoting
> Affects Versions: 3.0.1.Final
> Reporter: Jiri Ondrusek
> Assignee: Jiri Ondrusek
>
> User can add multiple {{http-connector}} s with the same socket binding:
> {code}[standalone@localhost:9990 /] /subsystem=remoting/http-connector=http-remoting-connector1:add(connector-ref=default, sasl-authentication-factory=application-sasl-authentication)
> {"outcome" => "success"}
> [standalone@localhost:9990 /] /subsystem=remoting/http-connector=http-remoting-connector2:add(connector-ref=default, sasl-authentication-factory=application-sasl-authentication)
> {"outcome" => "success"}{code}
> This is not possible with {{connector}}:
> {code}[standalone@localhost:9990 /] /subsystem=remoting/connector=dup1:add(sasl-authentication-factory=application-sasl-authentication, socket-binding=dup)
> {"outcome" => "success"}
> [standalone@localhost:9990 /] /subsystem=remoting/connector=dup2:add(sasl-authentication-factory=application-sasl-authentication, socket-binding=dup)
> {
> "outcome" => "failed",
> "failure-description" => \{"WFLYCTL0080: Failed services" => \{"jboss.remoting.server.dup2" => "WFLYRMT0004: Address already in use 127.0.0.1:12345
> Caused by: java.net.BindException: Address already in use"}},
> "rolled-back" => true
> }
> {code}
> The preferred behaviour should be consistent with {{connector}} - it makes no sense to have multiple connectors for single port as the client applications do not distinguish connectors by anything else than ports.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 9 months
[JBoss JIRA] (WFLY-7765) Application deployed as "singleton-deployment" in a cluster is not reachable for EJB calls under all circumstances
by Wolf-Dieter Fink (JIRA)
[ https://issues.jboss.org/browse/WFLY-7765?page=com.atlassian.jira.plugin.... ]
Wolf-Dieter Fink closed WFLY-7765.
----------------------------------
Resolution: Duplicate Issue
> Application deployed as "singleton-deployment" in a cluster is not reachable for EJB calls under all circumstances
> ------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-7765
> URL: https://issues.jboss.org/browse/WFLY-7765
> Project: WildFly
> Issue Type: Bug
> Components: EJB
> Reporter: Wolf-Dieter Fink
>
> If an application (singleton-deployment) is deployed into a cluster with more nodes than are initial in the list of the client it can happen that a node is elected to run this application which is not know initially by the client,
> or
> the node with the singleton application is shutted down or crash during runtime.
> In any case the EJB invocation will not work if a node is elected to run the deployment which is not part of the servers listed as initial connections at client side.
> With some BYTEMAN rules I checked that the client side ClusterTopologyMessageHandler and ClusterNodeRemovalHandler did not see more than the node which have the application deployed.
> If all nodes are added to the client jboss-ejb-client.properties the DeploymentNodeSelector is used for the first invocation, the cluster-view is received with one node.
> After this node is down the DNS will show another node and the cluster-view received afterwards onyl show this node.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 9 months
[JBoss JIRA] (WFLY-9340) Unignore MessagingClientTestCase.testMessagingClientUsingMessagingPort
by Miroslav Novak (JIRA)
[ https://issues.jboss.org/browse/WFLY-9340?page=com.atlassian.jira.plugin.... ]
Miroslav Novak updated WFLY-9340:
---------------------------------
Description:
Unignore MessagingClientTestCase#testMessagingClientUsingMessagingPort test. Smoke test whether JMS client can connect directly to messaging.
was:
Ignore MessagingClientTestCase#testMessagingClientUsingMessagingPort test. Smoke test whether JMS client can connect directly to messaging.
> Unignore MessagingClientTestCase.testMessagingClientUsingMessagingPort
> ----------------------------------------------------------------------
>
> Key: WFLY-9340
> URL: https://issues.jboss.org/browse/WFLY-9340
> Project: WildFly
> Issue Type: Task
> Components: Test Suite
> Affects Versions: 11.0.0.CR1
> Reporter: Miroslav Novak
> Assignee: Miroslav Novak
> Priority: Minor
> Fix For: 11.0.0.Final
>
>
> Unignore MessagingClientTestCase#testMessagingClientUsingMessagingPort test. Smoke test whether JMS client can connect directly to messaging.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 9 months
[JBoss JIRA] (JGRP-2218) New payload interface
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on JGRP-2218:
------------------------------------
I don't think we can produce a correct {{size()}} efficiently: any time we have a {{String}} key or value we'd have to duplicate most of the work of {{writeUTF}} in order to count the bytes. That's assuming we add {{size()}} to our externalizers, with the current API we'd have to serialize the object in a {{byte[]}} in order to compute the size. Could you make {{size()}} optional?
Also, I'm not convinced that we should preserve the buffer type on reading. E.g. with {{ObjectBuffer}}, if there is going to be a single ID for all the classloaders, it would have to serialize a reference to the classloader in the message, and I'm not sure how that would work. And does it ever make sense to read a {{CompositeBuffer}} and re-create all its sub-buffers on the receiver side? AFAICT the receiver's transport has to create a {{byte[]}} for the {{DataInput}} anyway, so it could just get a new buffer from the {{BufferManagement}} component and pass that up instead.
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectBuffer}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload management
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadManagement}} (or {{PayloadPool}} component for a new buffer. A naive implementation might create a new buffer every time, and more sophisticated one might use a pool of payloads.
> The {{PayloadManagement}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
8 years, 9 months