[JBoss JIRA] (JGRP-2218) New payload interface
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2218:
--------------------------------
If we created different message types rather than payloads, we could have the *same* memory allocation rate for byte[] array based messages than we have now! E.g.
{code:java}
public abstract class Msg {
protected Address dest_addr, src_addr;
protected volatile Header[] headers;
protected volatile short flags;
protected volatile byte transient_flags; // transient_flags is neither marshalled nor copied
}
{code}
{code:java}
public class ByteArrayMessage extends Msg {
protected int offset, length;
protected byte[] payload;
}
{code}
There could be other types of messages, e.g. {{ObjectMessage}} (equivalent to {{ObjectPayload}}), {{CompositeMessage}} and so on, but the {{ByteArrayMessage}} as sketched out above would have *exactly the same memory footprint (40 bytes)* as the current {{Message}} on {{master}}!
Perhaps people could create their own {{message}} subclasses (or implementations if {{Message}} were an interface!) and register them with a factory. The message concept is similar to what JMS provides.
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
> Attachments: jgrp-2218.jfr, master.jfr
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectPayload}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ByteArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> * {{InputStreamPayload}}: has a ref to an input stream and copies data from input- to output stream when marshalling
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> For fragmentation, method {{size()}} is crucial to determine whether a payload needs to be fragmented, or not. If, for example, a payload (e.g. an {{ObjectPayload}}) cannot determine the correct size, it may return {{-1}}. This leads to the {{ObjectPayload}} getting marshalled right away and getting wrapped into a {{ByteArrayPayload}}. So if {{size()}} cannot be determined, we have exactly the same behavior as what's currently done.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload factory
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadFactory}} component for a new buffer. A naive implementation might create a new buffer every time, but a more sophisticated one might use a pool of payloads.
> The {{PayloadFactory}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Symmetry
> When sending a {{CompositePayload}} of a 500 byte {{ByteArrayPayload}} and a 1000 byte {{NioDirectPayload}}, would we want to also get the same {{CompositePayload}} consisting of 2 payloads on the receiver side, or would we want to combine the 2 payloads into one and make the 2 payloads refer to the same combined byte[] array (or NIO buffer)? Should this be made configurable?
> h4. ObjectPayload
> If ObjectPayload cannot determine the size of the serialized data, it should return {{-1}}. This means that {{Message.setPayload(ObjectPayload)}} would right away serialize {{ObjectPayload}} into {{ByteArrayPayload}}.
> This means we do have the {{byte[]}} array creation (same as now), but for object payloads which do implement {{size()}} correctly, we could still do late serialization.
> h5. ObjectPayload and fragmentation
> {{FRAG3}} could decorate {{ObjectPayload}} with a fragmentation payload, which generates fragments on serialization and sends them down the stack.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0
[JBoss JIRA] (JGRP-2218) New payload interface
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Bela Ban edited comment on JGRP-2218 at 11/3/17 12:05 PM:
----------------------------------------------------------
If we created different message types rather than payloads, we could have the *same* memory allocation rate for byte[] array based messages than we have now! E.g.
{code:java}
public abstract class Msg {
protected Address dest_addr, src_addr;
protected volatile Header[] headers;
protected volatile short flags;
protected volatile byte transient_flags;
}
{code}
{code:java}
public class ByteArrayMessage extends Msg {
protected int offset, length;
protected byte[] payload;
}
{code}
There could be other types of messages, e.g. {{ObjectMessage}} (equivalent to {{ObjectPayload}}), {{CompositeMessage}} and so on, but the {{ByteArrayMessage}} as sketched out above would have *exactly the same memory footprint (40 bytes)* as the current {{Message}} on {{master}}!
Perhaps people could create their own {{message}} subclasses (or implementations if {{Message}} were an interface!) and register them with a factory. The message concept is similar to what JMS provides.
was (Author: belaban):
If we created different message types rather than payloads, we could have the *same* memory allocation rate for byte[] array based messages than we have now! E.g.
{code:java}
public abstract class Msg {
protected Address dest_addr, src_addr;
protected volatile Header[] headers;
protected volatile short flags;
protected volatile byte transient_flags; // transient_flags is neither marshalled nor copied
}
{code}
{code:java}
public class ByteArrayMessage extends Msg {
protected int offset, length;
protected byte[] payload;
}
{code}
There could be other types of messages, e.g. {{ObjectMessage}} (equivalent to {{ObjectPayload}}), {{CompositeMessage}} and so on, but the {{ByteArrayMessage}} as sketched out above would have *exactly the same memory footprint (40 bytes)* as the current {{Message}} on {{master}}!
Perhaps people could create their own {{message}} subclasses (or implementations if {{Message}} were an interface!) and register them with a factory. The message concept is similar to what JMS provides.
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
> Attachments: jgrp-2218.jfr, master.jfr
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectPayload}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ByteArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> * {{InputStreamPayload}}: has a ref to an input stream and copies data from input- to output stream when marshalling
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> For fragmentation, method {{size()}} is crucial to determine whether a payload needs to be fragmented, or not. If, for example, a payload (e.g. an {{ObjectPayload}}) cannot determine the correct size, it may return {{-1}}. This leads to the {{ObjectPayload}} getting marshalled right away and getting wrapped into a {{ByteArrayPayload}}. So if {{size()}} cannot be determined, we have exactly the same behavior as what's currently done.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload factory
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadFactory}} component for a new buffer. A naive implementation might create a new buffer every time, but a more sophisticated one might use a pool of payloads.
> The {{PayloadFactory}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Symmetry
> When sending a {{CompositePayload}} of a 500 byte {{ByteArrayPayload}} and a 1000 byte {{NioDirectPayload}}, would we want to also get the same {{CompositePayload}} consisting of 2 payloads on the receiver side, or would we want to combine the 2 payloads into one and make the 2 payloads refer to the same combined byte[] array (or NIO buffer)? Should this be made configurable?
> h4. ObjectPayload
> If ObjectPayload cannot determine the size of the serialized data, it should return {{-1}}. This means that {{Message.setPayload(ObjectPayload)}} would right away serialize {{ObjectPayload}} into {{ByteArrayPayload}}.
> This means we do have the {{byte[]}} array creation (same as now), but for object payloads which do implement {{size()}} correctly, we could still do late serialization.
> h5. ObjectPayload and fragmentation
> {{FRAG3}} could decorate {{ObjectPayload}} with a fragmentation payload, which generates fragments on serialization and sends them down the stack.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0
[JBoss JIRA] (JGRP-2218) New payload interface
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Bela Ban edited comment on JGRP-2218 at 11/3/17 12:00 PM:
----------------------------------------------------------
The {{UPerf}} test with 4 nodes shows slightly higher memory allocation for the payload use case (branch {{JGRP-2218}}) than on {{master}}, ca 15%:
* Master has memory allocated for TLAB: {{7.32 GB}} (allocation rate: {{124.90 MB/s}}) and JGRP-2218 has {{8.42 GB}} (allocation rate: {{143.79 MB/s}}).
* Master has memory allocated for objects: {{37.68 MB}} (allocation rate: {{643.09 kB/s}}) and JGRP-2218 has {{54.24 MB}} (allocation rate: {{926.04 kB/s}})
JFRs are attached.
I assume that the higher allocation is caused by the messages needing more memory: compared to 40 bytes for a {{Message}} on master, we need 32 bytes for the {{Message}} on {{JGRP-2218}}, and 24 bytes for a {{ByteArrayPayload}} (which is what was used by the test).
IMO this overhead can be optimized away, see my next comment.
was (Author: belaban):
The {{UPerf}} test with 4 nodes shows slightly higher memory allocation for the payload use case (branch {{JGRP-2218}}) than on {{master}}, ca 15%:
* Master has memory allocated for TLAB: {{7.32 GB}} (allocation rate: {{124.90 MB/s}}) and JGRP-2218 has {{8.42 GB}} (allocation rate: {{143.79 MB/s}}).
* Master has memory allocated for objects: {{37.68 MB}} (allocation rate: {{643.09 kB/s}}) and JGRP-2218 has {{54.24 MB}} (allocation rate: {{926.04 kB/s}})
JFRs are attached.
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
> Attachments: jgrp-2218.jfr, master.jfr
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectPayload}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ByteArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> * {{InputStreamPayload}}: has a ref to an input stream and copies data from input- to output stream when marshalling
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> For fragmentation, method {{size()}} is crucial to determine whether a payload needs to be fragmented, or not. If, for example, a payload (e.g. an {{ObjectPayload}}) cannot determine the correct size, it may return {{-1}}. This leads to the {{ObjectPayload}} getting marshalled right away and getting wrapped into a {{ByteArrayPayload}}. So if {{size()}} cannot be determined, we have exactly the same behavior as what's currently done.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload factory
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadFactory}} component for a new buffer. A naive implementation might create a new buffer every time, but a more sophisticated one might use a pool of payloads.
> The {{PayloadFactory}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Symmetry
> When sending a {{CompositePayload}} of a 500 byte {{ByteArrayPayload}} and a 1000 byte {{NioDirectPayload}}, would we want to also get the same {{CompositePayload}} consisting of 2 payloads on the receiver side, or would we want to combine the 2 payloads into one and make the 2 payloads refer to the same combined byte[] array (or NIO buffer)? Should this be made configurable?
> h4. ObjectPayload
> If ObjectPayload cannot determine the size of the serialized data, it should return {{-1}}. This means that {{Message.setPayload(ObjectPayload)}} would right away serialize {{ObjectPayload}} into {{ByteArrayPayload}}.
> This means we do have the {{byte[]}} array creation (same as now), but for object payloads which do implement {{size()}} correctly, we could still do late serialization.
> h5. ObjectPayload and fragmentation
> {{FRAG3}} could decorate {{ObjectPayload}} with a fragmentation payload, which generates fragments on serialization and sends them down the stack.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0
[JBoss JIRA] (WFCORE-3387) MBeanServer - not returning ManagedBean jboss.as:deployment=* during deployment
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-3387?page=com.atlassian.jira.plugi... ]
Brian Stansberry commented on WFCORE-3387:
------------------------------------------
I'll try and think more about the bigger questions, but FWIW I don't think we will fix the "bug" I described in my last comment. I can't see any particular benefit from doing so, while changing the current behavior might break people. To help prevent breaking that by mistake, I'll send up a PR now with a comment in the relevant code that explicitly states not to add the kind of verification check that would result in the InstanceNotFoundException.
> MBeanServer - not returning ManagedBean jboss.as:deployment=* during deployment
> -------------------------------------------------------------------------------
>
> Key: WFCORE-3387
> URL: https://issues.jboss.org/browse/WFCORE-3387
> Project: WildFly Core
> Issue Type: Bug
> Components: JMX
> Affects Versions: 3.0.8.Final
> Reporter: Nuno Godinho de Matos
> Priority: Minor
>
> In wildfly, there is a class of managed beans that are particularly useful to have available during application deployment.
> In partocular beans with Object name (e.g. jboss.as:deployment=someapp-war.war)
> This type of beans can for example sned a JMX Notification that the corresponding application has been deployed.
> While trying to refactor a mechanism that fires an "ApplicationDeployed" event, a mechanism that was blinding registering a Listener on a hard coded ObjectName stopped working consistently once I generalized to hunt for the generalized beans.
> The problem is the following.
> 1. HARD CODED APPROACH:
> If when an application is deploying, during the @Startup logic of an EJB I blindly do something like:
> {panel}
> MBeanServer mBeanServer = createMBeanServer();
> ObjectName targetObject = new ObjectName(objectName);
> mBeanServer.addNotificationListener(targetObject, listener, filter, handback);
> {panel}
> I can successfully register my Listener, without any problem.
> This will work 100% of the time, and after the deployment goes through I can send my notification to the application that the deployment is done.
> I just need to blindly register the listener on:
> jboss.as:deployment=someapp-war.war
> 2. PROBLEM - GENERALIZED APPORACH - QUERY MBEANS
> The Problem now is that once I try to generalize the mechanism, instead of directly doing something like jumping at the "registration", first I need to run a JMX query to hunt of beans:
> jboss.as:deployment=*
> Now I have the problem that I am coming out empty handed, no results.
> Better said, the current approach has two possible behaviors.
> Behavior 1 - Mechanism works:
> - Wildfly is stopped
> - on eclipse I drag and drop the WAR file to the APP server.
> - I start wildfly.
> In this case, the jboss.as:deployment=* will give me the deployed applications.
> The generic mechanism works.
> Behavior 2 - Query comes with no results
> - Wildfly is running
> - no deployments
> - I drag and drop the APP,
> then I get no results.
> Here is a Log snippet from the application when running on apporach 2.
> What you is multiple fluent wait queries.
> You have N tries loop where the query * and hard-coded object name is run. In both scenarios I get no results.
> But i still managed to register a listener If I want and I know the war name.
> {panel}
> ####2017-10-27 11:44:10,171 ThreadId:512 INFO [logger: .impl.jmx.AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,246 ThreadId:512 WARN [logger: .impl.jmx.AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,253 ThreadId:512 INFO [logger: .impl.jmx.AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,328 ThreadId:512 WARN [logger: .impl.jmx.AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,332 ThreadId:512 INFO [logger: .impl.jmx.AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,406 ThreadId:512 WARN [logger: .impl.jmx.AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,409 ThreadId:512 INFO [logger: .impl.jmx.AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,490 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,494 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,569 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,574 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,649 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,653 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,726 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,729 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,802 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,804 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,884 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,893 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,967 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:10,970 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,044 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,050 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,129 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,138 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,217 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,227 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,304 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,312 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,386 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,393 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,472 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,480 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,559 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,568 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,646 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,656 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,736 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,744 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,822 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,831 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,904 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,907 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,985 ThreadId:512 WARN [AbstractJmxAppDeploymentNotificationListenerRegistration] - Query jmx query: jboss.as:deployment=* yielded no results. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,991 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Using HARD_CODED_DIRTY_QUERY: jboss.as:deployment=someapp-war.war - returned: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,991 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - JMX Query to search for app deployments yielded: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,991 ThreadId:512 ERROR [AbstractJmxAppDeploymentNotificationListenerRegistration] - No WAR deployment could be found on the managed bean server. The applicatin runtimes that could be found were: [] <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,992 ThreadId:512 ERROR [AbstractJmxAppDeploymentNotificationListenerRegistration] - Going to add notification listerner using HARD CODED deployment JMX object name <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,992 ThreadId:512 INFO [AbstractJmxAppDeploymentNotificationListenerRegistration] - Going to register a jmx notification listener on the status of the deployed application: jboss.as:deployment=someapp-war.war <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,994 ThreadId:512 ERROR [AbstractJmxAppDeploymentNotificationListenerRegistration] - Skipping registration of NotificationLister on WAR application runtime, managed bean could not be found.This has the implication that the application ready event will not be fired after the deployment completes. <LogContext:Facade> <ServerService Thread Pool -- 339>
> ####2017-10-27 11:44:11,995 ThreadId:512 INFO [WildflyJmxAppDeploymentNotificationListenerRegistration] - <- WildflyJmxAppDeploymentNotificationListenerRegistration.registerNotificationLister {2092 ms} <LogContext:Facade> <ServerService Thread Pool -- 339>
> {panel}
> I can try to create a sample application to reproduce this.
> I would expect that before the @Startup logic is triggered on any EJB, I consistenly will have on the MBean server the deployment managemend bean for the WAR whose deployment is currently ongoing - regardless of whether I start the APP serer with the WAr already in or if I have just added the WAr file.
> And in effect, since the "registration mechanims of a listener always works - if you already know the WAR file name ... then it kind means that the Managed bean is in effect in the MBean serer but somehow not visible.
> Many thanks for any help on this.
> I will try to give you a sample application for analysis.
> Kindest regards.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0
[JBoss JIRA] (JGRP-2218) New payload interface
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Bela Ban updated JGRP-2218:
---------------------------
Attachment: master.jfr
jgrp-2218.jfr
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
> Attachments: jgrp-2218.jfr, master.jfr
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectPayload}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ByteArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> * {{InputStreamPayload}}: has a ref to an input stream and copies data from input- to output stream when marshalling
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> For fragmentation, method {{size()}} is crucial to determine whether a payload needs to be fragmented, or not. If, for example, a payload (e.g. an {{ObjectPayload}}) cannot determine the correct size, it may return {{-1}}. This leads to the {{ObjectPayload}} getting marshalled right away and getting wrapped into a {{ByteArrayPayload}}. So if {{size()}} cannot be determined, we have exactly the same behavior as what's currently done.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload factory
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadFactory}} component for a new buffer. A naive implementation might create a new buffer every time, but a more sophisticated one might use a pool of payloads.
> The {{PayloadFactory}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Symmetry
> When sending a {{CompositePayload}} of a 500 byte {{ByteArrayPayload}} and a 1000 byte {{NioDirectPayload}}, would we want to also get the same {{CompositePayload}} consisting of 2 payloads on the receiver side, or would we want to combine the 2 payloads into one and make the 2 payloads refer to the same combined byte[] array (or NIO buffer)? Should this be made configurable?
> h4. ObjectPayload
> If ObjectPayload cannot determine the size of the serialized data, it should return {{-1}}. This means that {{Message.setPayload(ObjectPayload)}} would right away serialize {{ObjectPayload}} into {{ByteArrayPayload}}.
> This means we do have the {{byte[]}} array creation (same as now), but for object payloads which do implement {{size()}} correctly, we could still do late serialization.
> h5. ObjectPayload and fragmentation
> {{FRAG3}} could decorate {{ObjectPayload}} with a fragmentation payload, which generates fragments on serialization and sends them down the stack.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0
[JBoss JIRA] (JGRP-2218) New payload interface
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Bela Ban edited comment on JGRP-2218 at 11/3/17 11:56 AM:
----------------------------------------------------------
The {{UPerf}} test with 4 nodes shows slightly higher memory allocation for the payload use case (branch {{JGRP-2218}}) than on {{master}}, ca 15%:
* Master has memory allocated for TLAB: {{7.32 GB}} (allocation rate: {{124.90 MB/s}}) and JGRP-2218 has {{8.42 GB}} (allocation rate: {{143.79 MB/s}}).
* Master has memory allocated for objects: {{37.68 MB}} (allocation rate: {{643.09 kB/s}}) and JGRP-2218 has {{54.24 MB}} (allocation rate: {{926.04 kB/s}})
JFRs are attached.
was (Author: belaban):
The {{UPerf}} test with 4 nodes shows slightly higher memory allocation for the payload use case (branch {{JGRP-2218}}) than on {{master}}, ca 15%:
* Master has memory allocated for TLAB: {{7.32 GB}} (allocation rate: {{124.90 MB/s}}) and JGRP-2218 has {{8.42 GB}} (allocation rate: {{143.79 MB/s}}).
* Master has memory allocated for objects: {{37.68 MB}} (allocation rate: {{643.09 kB/s}}) and JGRP-2218 has {{54.24 MB}} (allocation rate: {{926.04 kB/s}})
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
> Attachments: jgrp-2218.jfr, master.jfr
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectPayload}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ByteArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> * {{InputStreamPayload}}: has a ref to an input stream and copies data from input- to output stream when marshalling
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> For fragmentation, method {{size()}} is crucial to determine whether a payload needs to be fragmented, or not. If, for example, a payload (e.g. an {{ObjectPayload}}) cannot determine the correct size, it may return {{-1}}. This leads to the {{ObjectPayload}} getting marshalled right away and getting wrapped into a {{ByteArrayPayload}}. So if {{size()}} cannot be determined, we have exactly the same behavior as what's currently done.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload factory
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadFactory}} component for a new buffer. A naive implementation might create a new buffer every time, but a more sophisticated one might use a pool of payloads.
> The {{PayloadFactory}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Symmetry
> When sending a {{CompositePayload}} of a 500 byte {{ByteArrayPayload}} and a 1000 byte {{NioDirectPayload}}, would we want to also get the same {{CompositePayload}} consisting of 2 payloads on the receiver side, or would we want to combine the 2 payloads into one and make the 2 payloads refer to the same combined byte[] array (or NIO buffer)? Should this be made configurable?
> h4. ObjectPayload
> If ObjectPayload cannot determine the size of the serialized data, it should return {{-1}}. This means that {{Message.setPayload(ObjectPayload)}} would right away serialize {{ObjectPayload}} into {{ByteArrayPayload}}.
> This means we do have the {{byte[]}} array creation (same as now), but for object payloads which do implement {{size()}} correctly, we could still do late serialization.
> h5. ObjectPayload and fragmentation
> {{FRAG3}} could decorate {{ObjectPayload}} with a fragmentation payload, which generates fragments on serialization and sends them down the stack.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0
[JBoss JIRA] (JGRP-2218) New payload interface
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Bela Ban edited comment on JGRP-2218 at 11/3/17 11:55 AM:
----------------------------------------------------------
The {{UPerf}} test with 4 nodes shows slightly higher memory allocation for the payload use case (branch {{JGRP-2218}}) than on {{master}}, ca 15%:
* Master has memory allocated for TLAB: {{7.32 GB}} (allocation rate: {{124.90 MB/s}}) and JGRP-2218 has {{8.42 GB}} (allocation rate: {{143.79 MB/s}}).
* Master has memory allocated for objects: {{37.68 MB}} (allocation rate: {{643.09 kB/s}}) and JGRP-2218 has {{54.24 MB}} (allocation rate: {{926.04 kB/s}})
was (Author: belaban):
The {{UPerf}} test with 4 nodes shows slightly higher memory allocation for the payload use case (branch {{JGRP-2218}}) than on {{master}}, ca 15%:
* Master has memory allocated for TLAB: {{7.32 GB}} (allocation rate: {{124.90 MB/s}})and JGRP-2218 has {{8.42 GB}} (allocation rate: {{143.79 MB/s}}).
* Master has memory allocated for objects: {{37.68 MB}} (allocation rate: {{643.09 kB/s}}) and JGRP-2218 has {{54.24 MB}} (allocation rate: {{926.04 kB/s}})
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectPayload}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ByteArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> * {{InputStreamPayload}}: has a ref to an input stream and copies data from input- to output stream when marshalling
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> For fragmentation, method {{size()}} is crucial to determine whether a payload needs to be fragmented, or not. If, for example, a payload (e.g. an {{ObjectPayload}}) cannot determine the correct size, it may return {{-1}}. This leads to the {{ObjectPayload}} getting marshalled right away and getting wrapped into a {{ByteArrayPayload}}. So if {{size()}} cannot be determined, we have exactly the same behavior as what's currently done.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload factory
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadFactory}} component for a new buffer. A naive implementation might create a new buffer every time, but a more sophisticated one might use a pool of payloads.
> The {{PayloadFactory}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Symmetry
> When sending a {{CompositePayload}} of a 500 byte {{ByteArrayPayload}} and a 1000 byte {{NioDirectPayload}}, would we want to also get the same {{CompositePayload}} consisting of 2 payloads on the receiver side, or would we want to combine the 2 payloads into one and make the 2 payloads refer to the same combined byte[] array (or NIO buffer)? Should this be made configurable?
> h4. ObjectPayload
> If ObjectPayload cannot determine the size of the serialized data, it should return {{-1}}. This means that {{Message.setPayload(ObjectPayload)}} would right away serialize {{ObjectPayload}} into {{ByteArrayPayload}}.
> This means we do have the {{byte[]}} array creation (same as now), but for object payloads which do implement {{size()}} correctly, we could still do late serialization.
> h5. ObjectPayload and fragmentation
> {{FRAG3}} could decorate {{ObjectPayload}} with a fragmentation payload, which generates fragments on serialization and sends them down the stack.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0
[JBoss JIRA] (JGRP-2218) New payload interface
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-2218?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-2218:
--------------------------------
The {{UPerf}} test with 4 nodes shows slightly higher memory allocation for the payload use case (branch {{JGRP-2218}}) than on {{master}}, ca 15%:
* Master has memory allocated for TLAB: {{7.32 GB}} (allocation rate: {{124.90 MB/s}})and JGRP-2218 has {{8.42 GB}} (allocation rate: {{143.79 MB/s}}).
* Master has memory allocated for objects: {{37.68 MB}} (allocation rate: {{643.09 kB/s}}) and JGRP-2218 has {{54.24 MB}} (allocation rate: {{926.04 kB/s}})
> New payload interface
> ---------------------
>
> Key: JGRP-2218
> URL: https://issues.jboss.org/browse/JGRP-2218
> Project: JGroups
> Issue Type: Feature Request
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 5.0
>
>
> h3. Goal
> Change payload in {{Message}} from byte[] arrays to a {{Payload}} interface which can have multiple implementations.
> h3. Reason
> Currently, having to pass a byte[] array to a message leads to unnecessary copying:
> * When an application has a ref to an NIO (direct) {{ByteBuffer}}, the bytes in the byte buffer have to be copied into a byte[] array and then set in the message
> * When the application sends around byte[] arrays, but also wants to add some additional metadata, e.g. type (1000-byte requests/responses), it needs to create a new byte[] array of (say) 1001 bytes and copy the data (1000 bytes) plus the request type (1 byte) into the new copy. Example: {{MPerf}} and {{UPerf}}
> * When an object has to be sent (e.g. in Infinispan), the object has to be marshalled into a byte[] array (first allocation) and then added to the message. With the suggested {{ObjectPayload}} (below), marshalling of the object would occur late, and it would be marshalled directly into the output stream of the bundler, eliminating the byte[] array allocation made by the application.
> h3. Design
> Instead of copying, the application creates an instance of {{Payload}} and sets the payload in {{Message}}. The {{Payload}} is then passed all the way down into the transport where it is marshalled and sent. There can be a number of payload implementations, e.g.
> * {{ByteArrayPayload}}: wraps a byte[] array with an offset and length
> * {{NioDirectPayload}}: wraps an NIO direct {{ByteBuffer}}
> * {{NioHeapPayload}}: wraps an NIO heap-based {{ByteBuffer}}
> * {{CompositePayload}}: wraps multiple Buffers. E.g. type (1 byte) and data (1000 bytes) as described above
> * {{IntPayload}}: a single integer
> * {{ObjectPayload}}: has an Object and a ClassLoader (for reading), plus a Marshaller which know how to marshal the object, this allows for objects to be passed in payloads and they're only marshalled at the end (transport).
> * {{PartialPayload}}: a ref to a {{Payload}}, with an offset and length
> * {{InputStreamPayload}}: has a ref to an input stream and copies data from input- to output stream when marshalling
> The {{Payload}} interface has methods:
> * {{size()}}
> * {{writeTo(DataOutput)}}
> * {{readFrom(DataInput)}}
> * {{getInput()}}: this provides a {{DataInput}} stream for reading from the underlying payload
> and possibly also
> * {{acquire()}} and
> * {{release()}} (for ref-counting)
> * {{copy()}}
> Each payload impl has an ID and it should be possible to register new impls. A {{PayloadFactory}} maintains a mapping between IDs and impl classes.
> When marshalling a {{Payload}}, the ID is written first, followed by the payload's {{writeTo()}} method. When reading payloads, the {{PayloadFactory}} is used to create instances from IDs.
> h4. Fragmentation
> When fragmenting a buffer, the fragments are instances of {{PartialPayload}} which maintains an offset and length over an underlying payload. When marshalling a {{PartialPayload}}, only the part between offset and offset+length is written to the output stream.
> For fragmentation, method {{size()}} is crucial to determine whether a payload needs to be fragmented, or not. If, for example, a payload (e.g. an {{ObjectPayload}}) cannot determine the correct size, it may return {{-1}}. This leads to the {{ObjectPayload}} getting marshalled right away and getting wrapped into a {{ByteArrayPayload}}. So if {{size()}} cannot be determined, we have exactly the same behavior as what's currently done.
> h4. Reference counting
> If we implement ref-counting, then payloads can be reused as soon as the ref-count is 0. For example, when sending a message, the payload's ref-count could be incremented by the app calling {{acquire()}}. (Assuming the message is a unicast message), {{UNICAST3}} would increment the count to 2. This is needed because {{UNICAST3}} might have to retransmit the message if it was lost on the network, and meanwhile the payload cannot be reused (changed). The app calls {{release()}} when the {{JChannel.send()}} call returns, but the payload cannot be reused until {{UNICAST3}} calls {{release()}} as well. This will happen when an {{ACK}} for the given message has been received.
> h4. Payload factory
> When a request is received, the buffer is created from the bytes received on the network, based on the ID. This should be done by asking a {{PayloadFactory}} component for a new buffer. A naive implementation might create a new buffer every time, but a more sophisticated one might use a pool of payloads.
> The {{PayloadFactory}} instance could be replaced by one's own implementation; this allows for an application to control the lifecycle of payloads: thus the creation of buffers by the application and of payloads received over the network can be controlled by the same payload management impl.
> h4. Symmetry
> When sending a {{CompositePayload}} of a 500 byte {{ByteArrayPayload}} and a 1000 byte {{NioDirectPayload}}, would we want to also get the same {{CompositePayload}} consisting of 2 payloads on the receiver side, or would we want to combine the 2 payloads into one and make the 2 payloads refer to the same combined byte[] array (or NIO buffer)? Should this be made configurable?
> h4. ObjectPayload
> If ObjectPayload cannot determine the size of the serialized data, it should return {{-1}}. This means that {{Message.setPayload(ObjectPayload)}} would right away serialize {{ObjectPayload}} into {{ByteArrayPayload}}.
> This means we do have the {{byte[]}} array creation (same as now), but for object payloads which do implement {{size()}} correctly, we could still do late serialization.
> h5. ObjectPayload and fragmentation
> {{FRAG3}} could decorate {{ObjectPayload}} with a fragmentation payload, which generates fragments on serialization and sends them down the stack.
> h4. Misc
> * Since this issue includes API changes, the version will be 5.0
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0
[JBoss JIRA] (WFLY-9511) Wildfly stops working after java version upgrade
by Tomaz Cerar (JIRA)
[ https://issues.jboss.org/browse/WFLY-9511?page=com.atlassian.jira.plugin.... ]
Tomaz Cerar closed WFLY-9511.
-----------------------------
Assignee: (was: Jason Greene)
Resolution: Rejected
As Jaikiran said, this is a user question not a bug.
and easy solvable one by properly setting JAVA_HOME
> Wildfly stops working after java version upgrade
> ------------------------------------------------
>
> Key: WFLY-9511
> URL: https://issues.jboss.org/browse/WFLY-9511
> Project: WildFly
> Issue Type: Bug
> Affects Versions: 10.0.0.Final
> Reporter: Monika Sharma
>
> Our products installs wildfly with current version of java present on the client machine
> Client does an auto upgrade of java version and wildfly service stops. This is because the previous version of java on which wildfly was installed is no more there on that machine.
> Is there a way that the wildfly continue working even after java upgrade.
> Are there any new releases which would handle this issue
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0
[JBoss JIRA] (WFLY-3503) EJB method takes more time to return after each call
by subhendu sahu (JIRA)
[ https://issues.jboss.org/browse/WFLY-3503?page=com.atlassian.jira.plugin.... ]
subhendu sahu commented on WFLY-3503:
-------------------------------------
I am facing same problem in wildfly10. Is there any patch for this?
> EJB method takes more time to return after each call
> ----------------------------------------------------
>
> Key: WFLY-3503
> URL: https://issues.jboss.org/browse/WFLY-3503
> Project: WildFly
> Issue Type: Bug
> Components: EJB
> Affects Versions: JBoss AS7 7.2.0.Final
> Environment: windows 7
> Reporter: Cesar Tron-Lozai
> Assignee: David Lloyd
>
> I had this issue with JBoss EAP 6.1
> I have a method that loops for an arbitrary number of times and makes call to a simple get method from an stateless bean. Everytime the methods gets called, it takes more and more time to complete.
> Please look at the link on Stackoverflow for more details and code snipets
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 8 months
- 1
- 0