I have been struggling with this for awhile and I'm not sure if it's a problem with jboss or on the database side or the VPN.
Our configuration is as such:
We are connectin to a DB2 mainframe from Redhat Linux. The Linux machine has a VPN in place that encrypts the data as it comes across. We have DB2 Connect installed, but I am bypassing it by going directly to the MF through the VPN using type 4 DB2 driver.
I am running JBoss-as-7.1.3 (compiled from source). I am running CXF Restful Web Services to get data to pass on to another application. The error occurs in the DAOs. Things will run OK for awhile after I clean start JBoss 7, then we intermittently get the following types of errors:
10:21:58,719 ERROR [edu.berkeley.eas.enrollment.dao.classlist.TelebearsClassListDao] (http-/127.0.0.1:8380-74) +++ Exception occurred in stuprod.sect205f: CallableStatementCallback; SQL [{call stuprod.sect205f(?, ?, ?, ?, ?)}]; [jcc][t4][10335][10366][3.57.110] Invalid operation: Connection is closed. ERRORCODE=-4470, SQLSTATE=08003; nested exception is com.ibm.db2.jcc.am.io: [jcc][t4][10335][10366][3.57.110] Invalid operation: Connection is closed. ERRORCODE=-4470, SQLSTATE=08003: org.springframework.dao.DataAccessResourceFailureException: CallableStatementCallback; SQL [{call stuprod.sect205f(?, ?, ?, ?, ?)}]; [jcc][t4][10335][10366][3.57.110] Invalid operation: Connection is closed. ERRORCODE=-4470, SQLSTATE=08003; nested exception is com.ibm.db2.jcc.am.io: [jcc][t4][10335][10366][3.57.110] Invalid operation: Connection is closed. ERRORCODE=-4470, SQLSTATE=08003
Caused by: com.ibm.db2.jcc.am.io: [jcc][t4][10335][10366][3.57.110] Invalid operation: Connection is closed. ERRORCODE=-4470, SQLSTATE=08003
The errors happen more and more until all I get in the log are these errors. They happen in every stored procedure call (DAO), so it's no specific one.
I have attached the custom standalone.xml (enrollment-services.xml). I have tried many combinations of connection pool, threads (jca) etc and it always happens again. I am also attaching sample java classes for DAO and a service class.
This has become a real problem as this morning my production log has 5 million lines in it. Please help if you can.
Let me know if you need any more information.
Thanks, Ken