[JBoss JIRA] (JGRP-1994) TCP: remove send queues
by Bela Ban (JIRA)
Bela Ban created JGRP-1994:
------------------------------
Summary: TCP: remove send queues
Key: JGRP-1994
URL: https://issues.jboss.org/browse/JGRP-1994
Project: JGroups
Issue Type: Task
Reporter: Bela Ban
Assignee: Bela Ban
Fix For: 4.0
When {{TCP.use_send_queues}} is enabled, sending a message puts it in a queue from which a sender thread dequeues it and sends it.
This was a bad idea from the start, because
* We create 1 additional (sender) thread per peer. That's 999 threads *per member* if we have a cluster of 1000. Besides, more threads leads to increased context switching
* The queue only tapers over blocking on a TCP write, as it will fill up if the TCP write blocks. So the problem of a bloking write is only moved to a different part of the system, not eliminated.
* We have to *make a copy of every buffer* to be sent as buffers are reused -> unneeded memory allocation
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (JGRP-1989) Bundlers: reuse send buffer when transport == sync.
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1989?page=com.atlassian.jira.plugin.... ]
Bela Ban edited comment on JGRP-1989 at 12/14/15 7:15 AM:
----------------------------------------------------------
OK, so on the send side:
* Messages always go through the bundler, either as message batches or single messages (optimization for batch of 1)
** For bundled messages, a {{ByteArrayDataOutputStream}} with a capacity of {{max_bundle_size + MSG_OVERHEAD}} (the latter is space for version and flags) is created. The size of the output stream won't increase as no message bundle will ever be bigger than {{max_bundle_size}}.
** This buffer is reused for all messages bundles
** This is possible because - in all bundlers - a single thread sends the queued-up messages
* Single messages (marked as {{DONT_BUNDLE | OOB}}) will cause creation of a new buffer every single time
** This should not be a big issue as this combination is used very rarely
* There is no need for buffer pools in this case; a single buffer (in {{ByteArrayDataOutputStream}}) suffices
was (Author: belaban):
OK, so on the send side:
* Messages always go through the bundler, either as message batches or single messages (optimization for batch of 1)
** For bundled messages, a {{ByteArrayDataOutputStream}} with a capacity of {{max_bundle_size + MSG_OVERHEAD}} (the latter is space for version and flags) is created. The size of the output stream won't increase as no message bundle will ever be bigger than {{max_bundle_size}}
* Single messages (marked as {{DONT_BUNDLE | OOB}}) will cause creation of a new buffer every single time
** This should not be a big issue as this combination is used very rarely
* There is no need for buffer pools in this case; a single buffer (in {{ByteArrayDataOutputStream}}) suffices
> Bundlers: reuse send buffer when transport == sync.
> ---------------------------------------------------
>
> Key: JGRP-1989
> URL: https://issues.jboss.org/browse/JGRP-1989
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 3.6.7
>
>
> With the addition of {{TCP_NIO2}}, all bundlers now create new send buffers for every message (or message list). This generates a lot of memory allocations, perhaps it is better to revert this change for *synchronous transports* such as {{UDP}} and {{TCP}}, and still create new buffers for *asynchronous transports* such as {{TCP_NIO2}}.
> Synchronous transports guarantee a message has been put on the wire when {{TP.send()}} returns, whereas asynchronous transports may only have completed a partial write (so we cannot reuse the buffer).
> The code in the bundler should check for this, and copy if async or not copy if sync.
> Whether or not a transport is sync is determined by a new abstract method that needs to be overridden by every transport.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (JGRP-1989) Bundlers: reuse send buffer when transport == sync.
by Bela Ban (JIRA)
[ https://issues.jboss.org/browse/JGRP-1989?page=com.atlassian.jira.plugin.... ]
Bela Ban commented on JGRP-1989:
--------------------------------
OK, so on the send side:
* Messages always go through the bundler, either as message batches or single messages (optimization for batch of 1)
** For bundled messages, a {{ByteArrayDataOutputStream}} with a capacity of {{max_bundle_size + MSG_OVERHEAD}} (the latter is space for version and flags) is created. The size of the output stream won't increase as no message bundle will ever be bigger than {{max_bundle_size}}
* Single messages (marked as {{DONT_BUNDLE | OOB}}) will cause creation of a new buffer every single time
** This should not be a big issue as this combination is used very rarely
* There is no need for buffer pools in this case; a single buffer (in {{ByteArrayDataOutputStream}}) suffices
> Bundlers: reuse send buffer when transport == sync.
> ---------------------------------------------------
>
> Key: JGRP-1989
> URL: https://issues.jboss.org/browse/JGRP-1989
> Project: JGroups
> Issue Type: Enhancement
> Reporter: Bela Ban
> Assignee: Bela Ban
> Fix For: 3.6.7
>
>
> With the addition of {{TCP_NIO2}}, all bundlers now create new send buffers for every message (or message list). This generates a lot of memory allocations, perhaps it is better to revert this change for *synchronous transports* such as {{UDP}} and {{TCP}}, and still create new buffers for *asynchronous transports* such as {{TCP_NIO2}}.
> Synchronous transports guarantee a message has been put on the wire when {{TP.send()}} returns, whereas asynchronous transports may only have completed a partial write (so we cannot reuse the buffer).
> The code in the bundler should check for this, and copy if async or not copy if sync.
> Whether or not a transport is sync is determined by a new abstract method that needs to be overridden by every transport.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (WFCORE-761) Not possible to overlay non existing file in WAR
by Lin Gao (JIRA)
[ https://issues.jboss.org/browse/WFCORE-761?page=com.atlassian.jira.plugin... ]
Lin Gao reassigned WFCORE-761:
------------------------------
Assignee: Lin Gao (was: Stuart Douglas)
> Not possible to overlay non existing file in WAR
> ------------------------------------------------
>
> Key: WFCORE-761
> URL: https://issues.jboss.org/browse/WFCORE-761
> Project: WildFly Core
> Issue Type: Bug
> Components: Server
> Reporter: Bartosz Baranowski
> Assignee: Lin Gao
> Priority: Minor
>
> It is either bug in how deployments are treated or how overlay/vfs work.
> Steps to reproduce:
> 1. deploy undexploded war with jar inside
> 2. add overlay that will add non existing file in jar
> Result: exception:
> Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: JBAS018776: Failed to get content for deployment overlay WEB-INF/lib/overlayed.jar//META-INF/x/file.txt at WEB-INF/lib/overlayed.jar//META-INF/x/file.txt
> Caused by: java.io.FileNotFoundException: /content/shell.war/WEB-INF/lib/overlayed.jar/META-INF/x/file.txt"}}
> at org.jboss.as.test.integration.management.ManagementOperations.executeOperation(ManagementOperations.java:67)
> at org.jboss.as.test.integration.management.ManagementOperations.executeOperation(ManagementOperations.java:37)
> at org.jboss.as.test.integration.deployment.deploymentoverlay.jar.OverlayUtils.setupOverlay(OverlayUtils.java:76)
> at org.jboss.as.test.integration.deployment.deploymentoverlay.war.OverlayNonExistingResourceTestCase.testOverlay(OverlayNonExistingResourceTestCase.java:67)
> Expectation:
> should work. It actually does work, if war is really exploded or
> 'if(exploded)' part in overlay is removed from overlay processor and everything is handled via: https://github.com/stuartwdouglas/wildfly-core/blob/a75af9118c4062fafb899...
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (WFCORE-1217) CLI does not resolve multiple properties if one property is undefined
by Chao Wang (JIRA)
Chao Wang created WFCORE-1217:
---------------------------------
Summary: CLI does not resolve multiple properties if one property is undefined
Key: WFCORE-1217
URL: https://issues.jboss.org/browse/WFCORE-1217
Project: WildFly Core
Issue Type: Bug
Components: CLI
Affects Versions: 2.0.4.Final
Reporter: Chao Wang
Assignee: Chao Wang
https://bugzilla.redhat.com/show_bug.cgi?id=1289316 description:
{noformat}
Multiple property substitution is working with EAP 6.4.3+, however, if a variable amongst the multiple variables is empty or has no value, then the subsequent property in the CLI command is not substituted.
For example :
cat props.properties
-----------------------------------------
PROFILE-NAME=TestProfile
SERVER-INSTANCE-NUMBER=TestInstance
APP-VERSION=
VAR=test
-----------------------------------------
cat test.cli :
---------------------------------------
/host=master/server-config=${PROFILE-NAME}${APP-VERSION}${SERVER-INSTANCE-NUMBER}${VAR}:add(auto-start=true, group="${PROFILE-NAME}${APP-VERSION}-server-group")
---------------------------------------
and if I execute "./jboss-cli.sh --connect --file=test.cli --properties=props.properties" then I have the following in the host.xml ":
----------
...
<server name="TestProfile${SERVER-INSTANCE-NUMBER}test" group="TestProfile-server-group" auto-start="true"/>
...
-----------
Note APP-VERSION had no value, and so the subsequent SERVER-INSTANCE-NUMBER was not properly resolved
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months
[JBoss JIRA] (WFLY-5840) mod_cluster proxies doesn't accept expressions
by Radoslav Husar (JIRA)
[ https://issues.jboss.org/browse/WFLY-5840?page=com.atlassian.jira.plugin.... ]
Radoslav Husar edited comment on WFLY-5840 at 12/13/15 7:48 PM:
----------------------------------------------------------------
What you want to use, is this, run in a batch ideally:
{noformat}
[standalone@localhost:9990 /] /subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=advertise,value=false
{
"outcome" => "success",
"response-headers" => {
"operation-requires-reload" => true,
"process-state" => "reload-required"
}
}
[standalone@localhost:9990 /] /subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=proxies,value=[proxy1]
{
"outcome" => "success",
"response-headers" => {
"operation-requires-reload" => true,
"process-state" => "reload-required"
}
}
[standalone@localhost:9990 /] /socket-binding-group=standard-sockets/remote-destination-outbound-socket-binding=proxy1:add(host=${modcluster.host:modcluster},port=${modcluster.port:80}
{
"outcome" => "success",
"response-headers" => {"process-state" => "reload-required"}
}
[standalone@localhost:9990 /] :reload
{
"outcome" => "success",
"result" => undefined
}
{noformat}
PS: To understand an attribute, its best to read the description:
{noformat}
[standalone@localhost:9990 /] /subsystem=modcluster/mod-cluster-config=configuration/:read-resource-description
{
"outcome" => "success",
"result" => {
...
"description" => "List of proxies for mod_cluster to register with defined by outbound-socket-binding in socket-binding-group.",
{noformat}
was (Author: rhusar):
To understand an attribute, its best to read the description:
{noformat}
[standalone@localhost:9990 /] /subsystem=modcluster/mod-cluster-config=configuration/:read-resource-description
{
"outcome" => "success",
"result" => {
...
"description" => "List of proxies for mod_cluster to register with defined by outbound-socket-binding in socket-binding-group.",
{noformat}
What you want to use is this, run in a batch ideally:
{noformat}
[standalone@localhost:9990 /] /subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=advertise,value=false
{
"outcome" => "success",
"response-headers" => {
"operation-requires-reload" => true,
"process-state" => "reload-required"
}
}
[standalone@localhost:9990 /] /subsystem=modcluster/mod-cluster-config=configuration/:write-attribute(name=proxies,value=[proxy1]
{
"outcome" => "success",
"response-headers" => {
"operation-requires-reload" => true,
"process-state" => "reload-required"
}
}
[standalone@localhost:9990 /] /socket-binding-group=standard-sockets/remote-destination-outbound-socket-binding=proxy1:add(host=${modcluster.host:modcluster},port=${modcluster.port:80}
{
"outcome" => "success",
"response-headers" => {"process-state" => "reload-required"}
}
[standalone@localhost:9990 /] :reload
{
"outcome" => "success",
"result" => undefined
}
{noformat}
> mod_cluster proxies doesn't accept expressions
> ----------------------------------------------
>
> Key: WFLY-5840
> URL: https://issues.jboss.org/browse/WFLY-5840
> Project: WildFly
> Issue Type: Feature Request
> Components: Clustering
> Affects Versions: 9.0.2.Final
> Reporter: Rafael Benevides
> Assignee: Radoslav Husar
> Labels: mod_cluster
>
> proxy-list was deprecated in WFLY-457
> Now I'm forced to use proxies but it doesn't accept expressions.
> I want to create a customisable Docker image with where the modcluster ip can be configured at runtime
> I'd like that Wildfly image to accept the following command but
> {noformat}
> /subsystem=modcluster/mod-cluster-config=configuration:write-attribute(name=proxies,value=[${modcluster.host:modcluster}:${modcluster.port:80}])}}
> {noformat}
> but if fails with:{color:red} "Services that may be the cause:" => ["jboss.outbound-socket-binding.\"${modcluster.host:modcluster}:${modcluster.port:80}\""]{color}
> With
> {noformat}
> /subsystem=modcluster/:add-proxy(host=${modcluster.host:modcluster},port=80)
> {noformat}
> it fails with {color:red}WFLYMODCLS0016: No IP address could be resolved for the specified host of the proxy.{color}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
10 years, 5 months