3:49 p.m.
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to connect
to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:04 a.m.
New subject: [JBoss JIRA] Commented: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=comments#action_12364481 ]
Mladen Turk commented on JBWEB-82:
----------------------------------
Did you try adding connectionTimeout="600000" to the AJP/1.3 Connector in
server.xml?
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:14 a.m.
New subject: [JBoss JIRA] Commented: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=comments#action_12364556 ]
Devon Hillard commented on JBWEB-82:
------------------------------------
Based on the documentation available I thought that the default value was 60 seconds
(60000)? They weren't getting cleaned up at all. Did you mean 60000 or the 600000 in
your comment? Why would making the timeout 10X longer than default help?
I've solved this temporarily by moving to mod proxy_http connecting to the http
connector, and simply not using AJP at all. This setup seems to work ok. But I'd
still like to get the issue identified and ideally fixed.
Thanks!
Devon
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:28 p.m.
New subject: [JBoss JIRA] Commented: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=comments#action_12364599 ]
Mladen Turk commented on JBWEB-82:
----------------------------------
By default the connectionTimeout is infinite (note this is for AJP/1.3 connector not for
http), and yes I meant 600000 or 10 minutes.
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:18 p.m.
New subject: [JBoss JIRA] Commented: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=comments#action_12364601 ]
Devon Hillard commented on JBWEB-82:
------------------------------------
Ok. I can give that a try. Is this documented anywhere? Is the default different from
the AJP connector in JBoss 4.0.5?
If this is the normal behavior for having an unlimited timeout, then why is that the
default?
Thanks!
Devon
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6:29 a.m.
New subject: [JBoss JIRA] Commented: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=comments#action_12389282 ]
Robert Kirkbride commented on JBWEB-82:
---------------------------------------
I see the same behaviour with JBoss 4.2.2 on RHEL 5 using mod_jk 1.2.25 and Stock
Apache.
It can also be cleared by doing an 'apachectl graceful'.
My suspicion is that it's a tomcat/mod_jk problem.
We've had to revert back to JBoss 4.0.5.
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:43 a.m.
New subject: [JBoss JIRA] Resolved: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=all ]
Jean-Frederic Clere resolved JBWEB-82.
--------------------------------------
Resolution: Done
That is http://jira.jboss.com/jira/browse/JBPAPP-366.
To work around it use the jbossnative.
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
7:55 p.m.
New subject: [JBoss JIRA] Commented: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=comments#action_12390612 ]
Devon Hillard commented on JBWEB-82:
------------------------------------
I assume you mean to use the AJP listener in there? The only documentation link I could
find on that points here:
http://labs.jboss.com/jbossweb/config/config/conf03.html
Which is a 404 at the moment.
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
2 a.m.
New subject: [JBoss JIRA] Commented: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=comments#action_12391003 ]
Jean-Frederic Clere commented on JBWEB-82:
------------------------------------------
I have fixed the doc.
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:47 a.m.
New subject: [JBoss JIRA] Commented: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=comments#action_12396992 ]
Maty David commented on JBWEB-82:
---------------------------------
We are also facing the same problem of hung ajp connections after moving from jboss 4.0.5
to jboss 4.2.2.
Could you please elaborate on how can we do this workaround? how can we use the AJP
listener from jboss-native?
BTW, in what version of jboss-web this issue is resolved?
thanks,
Maty.
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
2:43 a.m.
New subject: [JBoss JIRA] Commented: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
[ http://jira.jboss.com/jira/browse/JBWEB-82?page=comments#action_12412556 ]
Jordi Gimￃﾃￂﾩnez commented on JBWEB-82:
---------------------------------------
Hi all,
It seems that Apache and JBoss are not configured for high-load websites by default. We
are using Apache 2.2.8 with JBoss 4.2.2 and were experiencing the same problems.
We followed this document's instructions
https://wiki.jboss.org/wiki/OptimalMod_jk1.2Configuration and after a two day test it
looks like it's 100% working now.
I encourage you to test this configuration and give some feedback whether it works for you
or not. Thanks to Devon Hillard for his off-the-forum feedback.
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to
connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just
upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last
night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error]
ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat
is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566):
(node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928):
(node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five,
although it seems to fluctuate. Most requests work, but many don't. There are NO
errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP
address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before
the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of
ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some
of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the
number of established connections down around 50-60. However, the connections that have
timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of
course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so
this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to
run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
6575
days inactive
6917
days old
10 comments
6 participants
participants (6)
-
Devon Hillard (JIRA) -
Jean-Frederic Clere (JIRA)
-
Jordi Gimￃﾃￂﾩnez (JIRA)
-
Maty David (JIRA)
-
Mladen Turk (JIRA)
-
Robert Kirkbride (JIRA)