[JBoss JIRA] Created: (JBWEB-82) JBossWeb leaves hung ajp connections with mod_jk
by Devon Hillard (JIRA)
JBossWeb leaves hung ajp connections with mod_jk
------------------------------------------------
Key: JBWEB-82
URL: http://jira.jboss.com/jira/browse/JBWEB-82
Project: JBoss Web
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: Tomcat Module
Environment: Apache 2.2.3, mod_jk 1.2, JBoss 4.2.0.GA
Reporter: Devon Hillard
Assigned To: Mladen Turk
I have a Debian (stable) server which is running Apache 2.2.3, and using mod_jk to connect to my application running within JBoss.
I have been running with this using JBoss 4.0.5 for many months, without issue. I just upgraded to JBoss 4.2.0. Things start off fine, however after running for a while (last night it took 6.5 hours) the mod_jk logs start showing these errors:
Code:
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_connection_tcp_get_message::jk_ajp_common.c (948
): (node1) can't receive the response message from tomcat, network problems or tomcat is down (*MyIP
*:8009), err=-104
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_get_reply::jk_ajp_common.c (1566): (node1) Tomca
t is down or refused connection. No response has been sent to the client (yet)
[Wed Jun 06 07:45:00 2007][29336:14016] [error] ajp_service::jk_ajp_common.c (1928): (node1) Connect
ing to tomcat failed. Tomcat is probably not started or is listening on the wrong port
These errors do NOT happen with each request. They happen perhaps one out of five, although it seems to fluctuate. Most requests work, but many don't. There are NO errors in the JBoss logs. I have tried using 127.0.0.1 and my actual non-loopback IP address and the same thing happens.
I can fix it by restarting JBoss, then everything works, for a few hours at least, before the problem starts up again.
-----------------
More information:
This appears to be related to open connections.
mod_jk was holding connections open to the JBoss web server on 8009, and the number of ESTABLISHED connections kept growing until tomcat started rejecting new connections. Some of the connections would get reused, but not that many.
I set the connection_pool_timeout in the mod_jk config to 10 seconds. This keeps the number of established connections down around 50-60. However, the connections that have timed out of the pool, appear to persist forever.
The connection from the mod_jk TO JBossWeb sticks in FIN_WAIT1 or FIN_WAIT2:
10.10.77.194:57587 10.10.77.194:8009 FIN_WAIT1
The connection view from JBossWeb gets stuck in CLOSE_WAIT:
10.10.77.194:8009 10.10.77.194:57587 CLOSE_WAIT
Once the number of connections hits 1024 (or thereabouts) everything falls apart, of course, and I have to kill -9 jboss and start it up again.
This never happened with JBoss 4.0.5. Apache and mod_jk haven't changed at all, so this has to be a change between Tomcat 5.5 or JBossWeb. I can't be the only person to run into this.....
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
18 years, 2 months
[JBoss JIRA] Created: (JBREM-979) Add invocation retry facility to http transport
by Ron Sigal (JIRA)
Add invocation retry facility to http transport
-----------------------------------------------
Key: JBREM-979
URL: http://jira.jboss.com/jira/browse/JBREM-979
Project: JBoss Remoting
Issue Type: Feature Request
Security Level: Public (Everyone can see)
Affects Versions: 2.4.0.CR2
Reporter: Ron Sigal
Assigned To: Ron Sigal
Fix For: 2.4.0.GA
The test org.jboss.test.remoting.transport.http.keep_alive.StressHTTPInvokerTestCase is failing occasionally with
testPostInvocation(org.jboss.test.remoting.transport.http.keep_alive.StressHTTPInvokerTestClient)java.lang.Exception: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
at org.jboss.test.remoting.transport.web.WebInvokerTestClient.testPostInvocationSub(WebInvokerTestClient.java:153)
at org.jboss.test.remoting.transport.web.WebInvokerTestClient.testPostInvocation(WebInvokerTestClient.java:54)
at org.jboss.test.remoting.transport.http.keep_alive.StressHTTPInvokerTestClient.testPostInvocation(StressHTTPInvokerTestClient.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at org.jboss.jrunit.harness.ServerTestHarness.executeTestSuite(ServerTestHarness.java:302)
at org.jboss.jrunit.harness.ServerTestHarness.access$000(ServerTestHarness.java:51)
at org.jboss.jrunit.harness.ServerTestHarness$2.run(ServerTestHarness.java:225)
Caused by: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:346)
at org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:146)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:177)
at org.jboss.remoting.Client.invoke(Client.java:1708)
at org.jboss.remoting.Client.invoke(Client.java:612)
at org.jboss.test.remoting.transport.web.WebInvokerTestClient.makeExceptionInvocation(WebInvokerTestClient.java:215)
at org.jboss.test.remoting.transport.web.WebInvokerTestClient.testPostInvocationSub(WebInvokerTestClient.java:148)
... 20 more
Caused by: java.io.IOException: Stream closed.
at java.net.PlainSocketImpl.available(PlainSocketImpl.java:428)
at java.net.SocketInputStream.available(SocketInputStream.java:217)
at java.io.BufferedInputStream.read(BufferedInputStream.java:321)
at sun.net.www.MeteredStream.read(MeteredStream.java:116)
at java.io.FilterInputStream.read(FilterInputStream.java:111)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2196)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2213)
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2226)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2694)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:761)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:277)
at org.jboss.remoting.loading.ObjectInputStreamWithClassLoader.<init>(ObjectInputStreamWithClassLoader.java:98)
at org.jboss.remoting.serialization.impl.java.JavaSerializationManager.createInput(JavaSerializationManager.java:54)
at org.jboss.remoting.marshal.serializable.SerializableUnMarshaller.getMarshallingStream(SerializableUnMarshaller.java:75)
at org.jboss.remoting.marshal.serializable.SerializableUnMarshaller.read(SerializableUnMarshaller.java:122)
at org.jboss.remoting.marshal.http.HTTPUnMarshaller.read(HTTPUnMarshaller.java:71)
at org.jboss.remoting.transport.http.HTTPClientInvoker.readResponse(HTTPClientInvoker.java:496)
at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:317)
... 26 more
It looks like the HttpURLConnection has returned a cached connection, the HTTPClientInvoker writes the invocation, and then it turns out that the server has closed the connection. It would be useful to add a retry facility parallel to the one in the socket transport.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
18 years, 2 months
[JBoss JIRA] Created: (HIBERNATE-95) JPA query does not work on Derby when more than one entity has a column named DATE
by Felipe Leme (JIRA)
JPA query does not work on Derby when more than one entity has a column named DATE
----------------------------------------------------------------------------------
Key: HIBERNATE-95
URL: http://jira.jboss.com/jira/browse/HIBERNATE-95
Project: Hibernate
Issue Type: Bug
Environment: Hibernate 3.2.6, Hibernate Annotations 3.3.0, Hibernate Entity Manager 3.3.1, Derby 10.4.1.3
Reporter: Felipe Leme
Assigned To: Steve Ebersole
I have a relationship that is more like this (sorry, didn't have time to provide a simpler scenario):
- MP has a field/column called date
- MPHistory also have the same field
- there is a 1-N relationship between MP -> MPHistory
- the relationship has a @OrderBy("date") clause
When Hibernate starts using H2, everything works fine. But if I use Derby, I get the following exception:
Caused by: java.sql.SQLException: Column name 'DATE' appears more than once in the result of the query expression.
The bad SQL is the following:
Hibernate: select distinct agententit0_.id as id1_0_, mps1_.id as id0_1_, history2_.id as id3_2_, agententit0_.connected as connected1_0_, agententit0_.jmx_name as jmx4_1_0_, agententit0_.status as status1_0_, agententit0_.host_id as host10_1_0_, agententit0_.host as host1_0_, agententit0_.local as local1_0_, agententit0_.port as port1_0_, agententit0_.type as type1_0_, mps1_.date as date0_1_, mps1_.name as name0_1_, mps1_.value as value0_1_, mps1_.frequency_type as frequency5_0_1_, mps1_.is_full_mode as is6_0_1_, mps1_.is_historical as is7_0_1_, mps1_.is_normal_mode as is8_0_1_, mps1_.tolerance as tolerance0_1_, mps1_.is_summary_mode as is10_0_1_, mps1_.value_class as value11_0_1_, mps1_.info_id as info12_0_1_, mps1_.obj_id as obj13_0__, mps1_.id as id0__, history2_.date as date3_2_, history2_.value as value3_2_, history2_.value_class as value4_3_2_, history2_.mp_id as mp5_1__, history2_.id as id1__ from ggs_objects agententit0_ left outer join mps mps1_ on agententit0_.id=mps1_.obj_id left outer join mps_history history2_ on mps1_.id=history2_.mp_id where agententit0_.type in (108, 8) order by date asc
More specifically, the final 'order by date asc' should be 'order by history2_.date asc'.
Doing some debugging, I found the problem at org.hibernate.sql.Template, method isFunctionOrKeyword (line 304), which returns:
"(".equals(nextToken) ||
KEYWORDS.contains(lcToken) ||
functionRegistry.hasFunction(lcToken) ||
dialect.getKeywords().contains(lcToken) ||
FUNCTION_KEYWORDS.contains(lcToken);
The 3rd check ( dialect.getKeywords().contains(lcToken) ) returns true for Derby, then the whole method returns true, which in turns does not prepend the "$PlaceHolder$ (history2_, in this case), in the query:
else if (
isIdentifier(token, dialect) &&
!isFunctionOrKeyword(lcToken, nextToken, dialect, functionRegistry) // <<<<< THIS IS THE ISSUE >>>>
) {
result.append(TEMPLATE)
.append('.')
.append( dialect.quote(token) );
}
For now, I just removed the @OrderBy from my query (it's not really important in my case), but it looks like the Template method logic is wrong in this case.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
18 years, 2 months