[JBoss JIRA] (MODCLUSTER-536) List of open files grows steadily during load test through mod_cluster
by Radoslav Husar (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-536?page=com.atlassian.jira.pl... ]
Radoslav Husar commented on MODCLUSTER-536:
-------------------------------------------
Can you look at the lsof what is actually leaking? Any chance the client is leaving the connections in CLOSE_WAIT state for instance?
If I am not mistaken, if you are testing on the same box, you are going to have 2 TCP connections for each request (Client <-> LB <-> Backend server) each consuming 4 file descriptors, so make sure you have the right settings to handle the corresponding load.
> List of open files grows steadily during load test through mod_cluster
> ----------------------------------------------------------------------
>
> Key: MODCLUSTER-536
> URL: https://issues.jboss.org/browse/MODCLUSTER-536
> Project: mod_cluster
> Issue Type: Bug
> Components: Core & Container Integration (Java)
> Affects Versions: 1.3.1.Final
> Environment: Wildfly10.0.0.Final
> mod_cluster-1.3.1.Final-linux2-x64-ssl
> CentOS7 (virtualbox)
> Reporter: Wayne Wang
> Assignee: Michal Karm Babacek
> Attachments: error_log, httpd-mpm.conf, httpd.conf, server.log, standalone-full-ha-snippet.xml
>
>
> I was able to configure wildfly 10 modcluster to work with Apache mod_cluster (1.3.1). However, when I was doing a load test, I found out that the test through web server eventually caused error in wildfly instance and I also saw error log in Apache web server
> The obvious error in wildfly instance is the so-called "java.net.SocketException: Too many files open". When I used the command lsop -u | grep TCP | wc -l, I can see the number grew steadily until the wildfly instance reported the error. This was when I sent requests through web server.
> However, when I sent the requests through wildfly instance (app server) directly, the number did not grow, and the app server can take a lot heavier load without this issue.
> The issue did not happen until many rounds of load tests were executed through web server. If I restart the web server, everything is working fine until I execute many rounds of load tests again
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months
[JBoss JIRA] (MODCLUSTER-531) Eliminate automagic
by Radoslav Husar (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-531?page=com.atlassian.jira.pl... ]
Radoslav Husar updated MODCLUSTER-531:
--------------------------------------
Description:
I am proposing we abandon automagic as much as possible.
h1. Problem
The main idea for automagic is to make deployment simple and easy to migrate/transfer and provide great user experience out of box. Unfortunately, this IMHO often backfires quite significantly resulting in configurations that are:
# insecure (listening on all interfaces, allow from all, default security key "changeme!")
# difficult to debug (not clear what is the actual configuration)
# unstable installations (seemingly unrelated aspects like adding an interface or connector break the config)
h1. Areas
There are several areas where automagic happens.
h5. Advertised address of the proxy
There were multiple bugs in the past, where 0.0.0.0 would be sent in the advertise mesages, now if its not explcit in the VirtualHost or passed in to ServerAdvertise, it automagically picks a non-local interface. Such configurations should be disallowed.
h5. Advertise interfaces
The interfaces are not explicit and advertise messages could be sent/received on more interfaces (related MODCLUSTER-487). This is also problematic when trying to move to DatagramChannel interface, which requires interfaces to be defined explicitly (MODCLUSTER-502). We can require this explicitly.
h5. Connector address
If bound to any-address, the address is inferred from the proxy connection as the local address. This is solved in the default WildFly configuration as its explicitly bound to a interface.
h5. Connector selection
This is solved in WildFly where selection is explicit. In tomcat this causes problems like MODCLUSTER-457 when WS requires http yet ajp is automatically selected by default. We can make this explicit.
h5. Route generation
Remove setJvmRoute from the SPI.
was:
I am proposing we abandon automagic as much as possible.
h1. Problem
The main idea for automagic is to make deployment simple and easy to migrate/transfer and provide great user experience out of box. Unfortunately, this IMHO often backfires quite significantly resulting in configurations that are:
# insecure (listening on all interfaces, allow from all, default security key "changeme!")
# difficult to debug (not clear what is the actual configuration)
# unstable installations (seemingly unrelated aspects like adding an interface or connector break the config)
h1. Areas
There are several areas where automagic happens.
h5. Advertised address of the proxy
There were multiple bugs in the past, where 0.0.0.0 would be sent in the advertise mesages, now if its not explcit in the VirtualHost or passed in to ServerAdvertise, it automagically picks a non-local interface. Such configurations should be disallowed.
h5. Advertise interfaces
The interfaces are not explicit and advertise messages could be sent/received on more interfaces (related MODCLUSTER-487). This is also problematic when trying to move to DatagramChannel interface, which requires interfaces to be defined explicitly (MODCLUSTER-502). We can require this explicitly.
h5. Connector address
If bound to any-address, the address is inferred from the proxy connection as the local address. This is solved in the default WildFly configuration as its explicitly bound to a interface.
h5. Connector selection
This is solved in WildFly where selection is explicit. In tomcat this causes problems like MODCLUSTER-457 when WS requires http yet ajp is automatically selected by default. We can make this explicit.
> Eliminate automagic
> -------------------
>
> Key: MODCLUSTER-531
> URL: https://issues.jboss.org/browse/MODCLUSTER-531
> Project: mod_cluster
> Issue Type: Enhancement
> Components: Core & Container Integration (Java), Documentation & Demos, Native (httpd modules)
> Reporter: Radoslav Husar
> Assignee: Radoslav Husar
>
> I am proposing we abandon automagic as much as possible.
> h1. Problem
> The main idea for automagic is to make deployment simple and easy to migrate/transfer and provide great user experience out of box. Unfortunately, this IMHO often backfires quite significantly resulting in configurations that are:
> # insecure (listening on all interfaces, allow from all, default security key "changeme!")
> # difficult to debug (not clear what is the actual configuration)
> # unstable installations (seemingly unrelated aspects like adding an interface or connector break the config)
> h1. Areas
> There are several areas where automagic happens.
> h5. Advertised address of the proxy
> There were multiple bugs in the past, where 0.0.0.0 would be sent in the advertise mesages, now if its not explcit in the VirtualHost or passed in to ServerAdvertise, it automagically picks a non-local interface. Such configurations should be disallowed.
> h5. Advertise interfaces
> The interfaces are not explicit and advertise messages could be sent/received on more interfaces (related MODCLUSTER-487). This is also problematic when trying to move to DatagramChannel interface, which requires interfaces to be defined explicitly (MODCLUSTER-502). We can require this explicitly.
> h5. Connector address
> If bound to any-address, the address is inferred from the proxy connection as the local address. This is solved in the default WildFly configuration as its explicitly bound to a interface.
> h5. Connector selection
> This is solved in WildFly where selection is explicit. In tomcat this causes problems like MODCLUSTER-457 when WS requires http yet ajp is automatically selected by default. We can make this explicit.
> h5. Route generation
> Remove setJvmRoute from the SPI.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months
[JBoss JIRA] (MODCLUSTER-536) List of open files grows steadily during load test through mod_cluster
by Wayne Wang (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-536?page=com.atlassian.jira.pl... ]
Wayne Wang commented on MODCLUSTER-536:
---------------------------------------
I found the server produced error and the lsof showed over 3000, so I set the soft limit to 4096 and hard limit on 8192. However, this issue persisted.
for example, for the machine starting up httpd,
daemon soft nofile 4096
daemon hard nofile 8192
Note, the machine that starting up httpd will spawn httpd processes under user daemon.
> List of open files grows steadily during load test through mod_cluster
> ----------------------------------------------------------------------
>
> Key: MODCLUSTER-536
> URL: https://issues.jboss.org/browse/MODCLUSTER-536
> Project: mod_cluster
> Issue Type: Bug
> Components: Core & Container Integration (Java)
> Affects Versions: 1.3.1.Final
> Environment: Wildfly10.0.0.Final
> mod_cluster-1.3.1.Final-linux2-x64-ssl
> CentOS7 (virtualbox)
> Reporter: Wayne Wang
> Assignee: Michal Karm Babacek
> Attachments: error_log, httpd-mpm.conf, httpd.conf, server.log, standalone-full-ha-snippet.xml
>
>
> I was able to configure wildfly 10 modcluster to work with Apache mod_cluster (1.3.1). However, when I was doing a load test, I found out that the test through web server eventually caused error in wildfly instance and I also saw error log in Apache web server
> The obvious error in wildfly instance is the so-called "java.net.SocketException: Too many files open". When I used the command lsop -u | grep TCP | wc -l, I can see the number grew steadily until the wildfly instance reported the error. This was when I sent requests through web server.
> However, when I sent the requests through wildfly instance (app server) directly, the number did not grow, and the app server can take a lot heavier load without this issue.
> The issue did not happen until many rounds of load tests were executed through web server. If I restart the web server, everything is working fine until I execute many rounds of load tests again
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months
[JBoss JIRA] (MODCLUSTER-536) List of open files grows steadily during load test through mod_cluster
by Wayne Wang (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-536?page=com.atlassian.jira.pl... ]
Wayne Wang edited comment on MODCLUSTER-536 at 8/31/16 11:35 AM:
-----------------------------------------------------------------
Additional information:
I could shut down the instance in two different ways during load tests
(1) After Jmeter send button is back (all requests were sent to a specific wildfly instance for processing)
(2) Before Jmeter send button is back, and the rest of requests need to be routed to the soon-to-be active wildfly instance that is currently inactive.
I found one round of load test with the second way of shutting down the wildfly instance will cause the "Too many open files" issue in the next round of load test. If I use the first way of shutting down the wildfly instance, it needs three times of load tests to replicate the issue.
So it looks like shutting down a wildfly instance may be one potential cause of the issue. In addition, the activation of the standby instance takes some time, and the rest of requests will go nowhere during that period of time until the standby instance becomes active. I am not sure if this can bring additional issue
was (Author: wwang2016):
Additional information:
I could shut down the instance in two different ways during load tests
(1) After Jmeter send button is back (all requests were sent to a specific wildfly instance for processing)
(2) Before Jmeter send button is back, and the rest of requests need to be routed to the soon-to-be active wildfly instance that is currently inactive.
I found one round of load test with the second way of shutting down the wildfly instance will cause the "Too many open files" issue in the next round of load test. If I use the first way of shutting down the wildfly instance, it needs three times of load tests to replicate the issue.
> List of open files grows steadily during load test through mod_cluster
> ----------------------------------------------------------------------
>
> Key: MODCLUSTER-536
> URL: https://issues.jboss.org/browse/MODCLUSTER-536
> Project: mod_cluster
> Issue Type: Bug
> Components: Core & Container Integration (Java)
> Affects Versions: 1.3.1.Final
> Environment: Wildfly10.0.0.Final
> mod_cluster-1.3.1.Final-linux2-x64-ssl
> CentOS7 (virtualbox)
> Reporter: Wayne Wang
> Assignee: Michal Karm Babacek
> Attachments: error_log, httpd-mpm.conf, httpd.conf, server.log, standalone-full-ha-snippet.xml
>
>
> I was able to configure wildfly 10 modcluster to work with Apache mod_cluster (1.3.1). However, when I was doing a load test, I found out that the test through web server eventually caused error in wildfly instance and I also saw error log in Apache web server
> The obvious error in wildfly instance is the so-called "java.net.SocketException: Too many files open". When I used the command lsop -u | grep TCP | wc -l, I can see the number grew steadily until the wildfly instance reported the error. This was when I sent requests through web server.
> However, when I sent the requests through wildfly instance (app server) directly, the number did not grow, and the app server can take a lot heavier load without this issue.
> The issue did not happen until many rounds of load tests were executed through web server. If I restart the web server, everything is working fine until I execute many rounds of load tests again
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months
[JBoss JIRA] (MODCLUSTER-536) List of open files grows steadily during load test through mod_cluster
by Wayne Wang (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-536?page=com.atlassian.jira.pl... ]
Wayne Wang commented on MODCLUSTER-536:
---------------------------------------
Additional information:
I could shut down the instance in two different ways during load tests
(1) After Jmeter send button is back (all requests were sent to a specific wildfly instance for processing)
(2) Before Jmeter send button is back, and the rest of requests need to be routed to the soon-to-be active wildfly instance that is currently inactive.
I found one round of load test with the second way of shutting down the wildfly instance will cause the "Too many open files" issue in the next round of load test. If I use the first way of shutting down the wildfly instance, it needs three times of load tests to replicate the issue.
> List of open files grows steadily during load test through mod_cluster
> ----------------------------------------------------------------------
>
> Key: MODCLUSTER-536
> URL: https://issues.jboss.org/browse/MODCLUSTER-536
> Project: mod_cluster
> Issue Type: Bug
> Components: Core & Container Integration (Java)
> Affects Versions: 1.3.1.Final
> Environment: Wildfly10.0.0.Final
> mod_cluster-1.3.1.Final-linux2-x64-ssl
> CentOS7 (virtualbox)
> Reporter: Wayne Wang
> Assignee: Michal Karm Babacek
> Attachments: error_log, httpd-mpm.conf, httpd.conf, server.log, standalone-full-ha-snippet.xml
>
>
> I was able to configure wildfly 10 modcluster to work with Apache mod_cluster (1.3.1). However, when I was doing a load test, I found out that the test through web server eventually caused error in wildfly instance and I also saw error log in Apache web server
> The obvious error in wildfly instance is the so-called "java.net.SocketException: Too many files open". When I used the command lsop -u | grep TCP | wc -l, I can see the number grew steadily until the wildfly instance reported the error. This was when I sent requests through web server.
> However, when I sent the requests through wildfly instance (app server) directly, the number did not grow, and the app server can take a lot heavier load without this issue.
> The issue did not happen until many rounds of load tests were executed through web server. If I restart the web server, everything is working fine until I execute many rounds of load tests again
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months
[JBoss JIRA] (MODCLUSTER-536) List of open files grows steadily during load test through mod_cluster
by Wayne Wang (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-536?page=com.atlassian.jira.pl... ]
Wayne Wang commented on MODCLUSTER-536:
---------------------------------------
>From the latest load tests where requests were sent through web server (httpd) and routed to wildfly10.
I only test the performance, and not testing HA singleton and fail-over. This time, I have NOT seen the outcome of lsof to grow steadily. I tested 14 times, and the servers look strong.
I re-tested the HA singleton configuration to record each shutdown of wildfly instance, and I was able to replicate the issue again with only 4 times of load test.
So it looks like the shutdown of wildfly instance (behind web server) is related to the issue of growing number of open files in both vm for wildfly instance and web server. Is there any way to get around it other than restart the httpd?
> List of open files grows steadily during load test through mod_cluster
> ----------------------------------------------------------------------
>
> Key: MODCLUSTER-536
> URL: https://issues.jboss.org/browse/MODCLUSTER-536
> Project: mod_cluster
> Issue Type: Bug
> Components: Core & Container Integration (Java)
> Affects Versions: 1.3.1.Final
> Environment: Wildfly10.0.0.Final
> mod_cluster-1.3.1.Final-linux2-x64-ssl
> CentOS7 (virtualbox)
> Reporter: Wayne Wang
> Assignee: Michal Karm Babacek
> Attachments: error_log, httpd-mpm.conf, httpd.conf, server.log, standalone-full-ha-snippet.xml
>
>
> I was able to configure wildfly 10 modcluster to work with Apache mod_cluster (1.3.1). However, when I was doing a load test, I found out that the test through web server eventually caused error in wildfly instance and I also saw error log in Apache web server
> The obvious error in wildfly instance is the so-called "java.net.SocketException: Too many files open". When I used the command lsop -u | grep TCP | wc -l, I can see the number grew steadily until the wildfly instance reported the error. This was when I sent requests through web server.
> However, when I sent the requests through wildfly instance (app server) directly, the number did not grow, and the app server can take a lot heavier load without this issue.
> The issue did not happen until many rounds of load tests were executed through web server. If I restart the web server, everything is working fine until I execute many rounds of load tests again
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months
[JBoss JIRA] (MODCLUSTER-536) List of open files grows steadily during load test through mod_cluster
by Wayne Wang (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-536?page=com.atlassian.jira.pl... ]
Wayne Wang commented on MODCLUSTER-536:
---------------------------------------
I have wildfly running in one virtualbox vm, and httpd running in another virtualbox vm. They are running under different users (wildfly running under a user and httpd running under root).
I would like to mention that this is a HA singleton configuration that I am testing. During load tests, I would shut down the active wildfly instance, and it will fail over to another standby instance. I saw the issue after a couple of such tests,
I am not sure if these tests could introduce issues to httpd connections with wildfly instance. However, these test cases are expected scenarios in production. For example, we need to shut down the active instance and do maintenance work and let the application fail over to another standby instance.
Nevertheless, I will try to test the scenario where the wildfly instance is not shutdown, and I load test through web server (httpd), and see if there is any issue.
> List of open files grows steadily during load test through mod_cluster
> ----------------------------------------------------------------------
>
> Key: MODCLUSTER-536
> URL: https://issues.jboss.org/browse/MODCLUSTER-536
> Project: mod_cluster
> Issue Type: Bug
> Components: Core & Container Integration (Java)
> Affects Versions: 1.3.1.Final
> Environment: Wildfly10.0.0.Final
> mod_cluster-1.3.1.Final-linux2-x64-ssl
> CentOS7 (virtualbox)
> Reporter: Wayne Wang
> Assignee: Michal Karm Babacek
> Attachments: error_log, httpd-mpm.conf, httpd.conf, server.log, standalone-full-ha-snippet.xml
>
>
> I was able to configure wildfly 10 modcluster to work with Apache mod_cluster (1.3.1). However, when I was doing a load test, I found out that the test through web server eventually caused error in wildfly instance and I also saw error log in Apache web server
> The obvious error in wildfly instance is the so-called "java.net.SocketException: Too many files open". When I used the command lsop -u | grep TCP | wc -l, I can see the number grew steadily until the wildfly instance reported the error. This was when I sent requests through web server.
> However, when I sent the requests through wildfly instance (app server) directly, the number did not grow, and the app server can take a lot heavier load without this issue.
> The issue did not happen until many rounds of load tests were executed through web server. If I restart the web server, everything is working fine until I execute many rounds of load tests again
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months
[JBoss JIRA] (MODCLUSTER-536) List of open files grows steadily during load test through mod_cluster
by Jean-Frederic Clere (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-536?page=com.atlassian.jira.pl... ]
Jean-Frederic Clere edited comment on MODCLUSTER-536 at 8/31/16 4:34 AM:
-------------------------------------------------------------------------
mod_cluster persists the connections between httpd and wilfly to reuse them. Basically you need to make sure the user daemon has a ulimit -n big enough to deal with the number of connection. once the max file is reached httpd might not cover well :-(
Looking in httpd-mpm.conf (MaxRequestWorkers 400) basically you are going to have at least 800 files open, very neat to 1024 default limit.
Is wildfly runned in the same user?
was (Author: jfclere):
mod_cluster persists the connection between httpd and wilfly to reuse them. Basically you need to make sure the user daemon has a ulimit -n big enough to deal with the number of connection. once the max file is reached httpd might not cover well :-(
Looking in httpd-mpm.conf (MaxRequestWorkers 400) basically you are going to have at least 800 files open, very neat to 1024 default limit.
Is wildfly runned in the same user?
> List of open files grows steadily during load test through mod_cluster
> ----------------------------------------------------------------------
>
> Key: MODCLUSTER-536
> URL: https://issues.jboss.org/browse/MODCLUSTER-536
> Project: mod_cluster
> Issue Type: Bug
> Components: Core & Container Integration (Java)
> Affects Versions: 1.3.1.Final
> Environment: Wildfly10.0.0.Final
> mod_cluster-1.3.1.Final-linux2-x64-ssl
> CentOS7 (virtualbox)
> Reporter: Wayne Wang
> Assignee: Michal Karm Babacek
> Attachments: error_log, httpd-mpm.conf, httpd.conf, server.log, standalone-full-ha-snippet.xml
>
>
> I was able to configure wildfly 10 modcluster to work with Apache mod_cluster (1.3.1). However, when I was doing a load test, I found out that the test through web server eventually caused error in wildfly instance and I also saw error log in Apache web server
> The obvious error in wildfly instance is the so-called "java.net.SocketException: Too many files open". When I used the command lsop -u | grep TCP | wc -l, I can see the number grew steadily until the wildfly instance reported the error. This was when I sent requests through web server.
> However, when I sent the requests through wildfly instance (app server) directly, the number did not grow, and the app server can take a lot heavier load without this issue.
> The issue did not happen until many rounds of load tests were executed through web server. If I restart the web server, everything is working fine until I execute many rounds of load tests again
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months
[JBoss JIRA] (MODCLUSTER-536) List of open files grows steadily during load test through mod_cluster
by Jean-Frederic Clere (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-536?page=com.atlassian.jira.pl... ]
Jean-Frederic Clere commented on MODCLUSTER-536:
------------------------------------------------
mod_cluster persists the connection between httpd and wilfly to reuse them. Basically you need to make sure the user daemon has a ulimit -n big enough to deal with the number of connection. once the max file is reached httpd might not cover well :-(
Looking in httpd-mpm.conf (MaxRequestWorkers 400) basically you are going to have at least 800 files open, very neat to 1024 default limit.
Is wildfly runned in the same user?
> List of open files grows steadily during load test through mod_cluster
> ----------------------------------------------------------------------
>
> Key: MODCLUSTER-536
> URL: https://issues.jboss.org/browse/MODCLUSTER-536
> Project: mod_cluster
> Issue Type: Bug
> Components: Core & Container Integration (Java)
> Affects Versions: 1.3.1.Final
> Environment: Wildfly10.0.0.Final
> mod_cluster-1.3.1.Final-linux2-x64-ssl
> CentOS7 (virtualbox)
> Reporter: Wayne Wang
> Assignee: Michal Karm Babacek
> Attachments: error_log, httpd-mpm.conf, httpd.conf, server.log, standalone-full-ha-snippet.xml
>
>
> I was able to configure wildfly 10 modcluster to work with Apache mod_cluster (1.3.1). However, when I was doing a load test, I found out that the test through web server eventually caused error in wildfly instance and I also saw error log in Apache web server
> The obvious error in wildfly instance is the so-called "java.net.SocketException: Too many files open". When I used the command lsop -u | grep TCP | wc -l, I can see the number grew steadily until the wildfly instance reported the error. This was when I sent requests through web server.
> However, when I sent the requests through wildfly instance (app server) directly, the number did not grow, and the app server can take a lot heavier load without this issue.
> The issue did not happen until many rounds of load tests were executed through web server. If I restart the web server, everything is working fine until I execute many rounds of load tests again
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months
[JBoss JIRA] (MODCLUSTER-536) List of open files grows steadily during load test through mod_cluster
by Wayne Wang (JIRA)
[ https://issues.jboss.org/browse/MODCLUSTER-536?page=com.atlassian.jira.pl... ]
Wayne Wang commented on MODCLUSTER-536:
---------------------------------------
Additional comments:
(1) When I tested through App server (wildfly10) directly, the application can handle a lot heavier load without error
(2) Once a load test through web server encounter error (lsof showed a big number), the web server (mod_cluster) will not be in good state even when lsof showed a small number after an extended period of time. It has to be restarted to handle a load test. Again, it can not handle many rounds of load tests.
> List of open files grows steadily during load test through mod_cluster
> ----------------------------------------------------------------------
>
> Key: MODCLUSTER-536
> URL: https://issues.jboss.org/browse/MODCLUSTER-536
> Project: mod_cluster
> Issue Type: Bug
> Components: Core & Container Integration (Java)
> Affects Versions: 1.3.1.Final
> Environment: Wildfly10.0.0.Final
> mod_cluster-1.3.1.Final-linux2-x64-ssl
> CentOS7 (virtualbox)
> Reporter: Wayne Wang
> Assignee: Michal Karm Babacek
> Attachments: error_log, httpd-mpm.conf, httpd.conf, server.log, standalone-full-ha-snippet.xml
>
>
> I was able to configure wildfly 10 modcluster to work with Apache mod_cluster (1.3.1). However, when I was doing a load test, I found out that the test through web server eventually caused error in wildfly instance and I also saw error log in Apache web server
> The obvious error in wildfly instance is the so-called "java.net.SocketException: Too many files open". When I used the command lsop -u | grep TCP | wc -l, I can see the number grew steadily until the wildfly instance reported the error. This was when I sent requests through web server.
> However, when I sent the requests through wildfly instance (app server) directly, the number did not grow, and the app server can take a lot heavier load without this issue.
> The issue did not happen until many rounds of load tests were executed through web server. If I restart the web server, everything is working fine until I execute many rounds of load tests again
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 3 months