[JBoss JIRA] (WFLY-9657) NullPointerException when trying to get all job executions
by Bartosz Baranowski (JIRA)
[ https://issues.jboss.org/browse/WFLY-9657?page=com.atlassian.jira.plugin.... ]
Bartosz Baranowski reassigned WFLY-9657:
----------------------------------------
Assignee: Bartosz Baranowski (was: James Perkins)
> NullPointerException when trying to get all job executions
> ----------------------------------------------------------
>
> Key: WFLY-9657
> URL: https://issues.jboss.org/browse/WFLY-9657
> Project: WildFly
> Issue Type: Bug
> Components: Batch
> Affects Versions: 11.0.0.Final
> Reporter: Cheng Fang
> Assignee: Bartosz Baranowski
>
> In jberet-ui, the job executions page by default displays all known job executions in the batch runtime. When running in WildFly 11, clicking the jberet-ui job executions page shows no job execution, and in server.log, there is the NPE:
> {code:text}
> Caused by: java.lang.NullPointerException
> at org.wildfly.extension.batch.jberet.deployment.JobOperatorService.getJobExecutions(JobOperatorService.java:295)
> at org.jberet.operations.DelegatingJobOperator.getJobExecutions(DelegatingJobOperator.java:92)
> at org.jberet.rest.service.JobService.getJobExecutions(JobService.java:113)
> at org.jberet.rest.resource.JobExecutionResource.getJobExecutions(JobExecutionResource.java:64)
> at org.jberet.rest.resource.JobExecutionResource$Proxy$_$$_WeldClientProxy.getJobExecutions(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140)
> at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)
> at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)
> at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236)
> at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406)
> ... 43 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (WFLY-10749) Artemis with JDBC store on slow or high latency database crashed on: Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
by Miroslav Novak (JIRA)
[ https://issues.jboss.org/browse/WFLY-10749?page=com.atlassian.jira.plugin... ]
Miroslav Novak commented on WFLY-10749:
---------------------------------------
[~martyn-taylor] Could you clone this to AMQ jira tracker as well, please? As code base for JDBC store is basically the same in Artemis 1.5.x and 2.x, this issue affects AMQ 7 as well.
> Artemis with JDBC store on slow or high latency database crashed on: Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-10749
> URL: https://issues.jboss.org/browse/WFLY-10749
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Affects Versions: 12.0.0.Final, 13.0.0.Final
> Reporter: Miroslav Novak
> Assignee: Martyn Taylor
> Priority: Critical
>
> Artemis configured with JDBC store is not robust if database is slow/under high load and when there is higher latency to database. Artemis has tendency to shutdown itself on "Critical IO Error" under really low load. It is visible when EAP QE moved Jenkins to Central CI where is higher latency to used Oracle12cR2 database. In following test scenario:
> * Start 2 WF/EAP server in cluster. There is queue InQueue and OutQueue deployed on both of the servers.
> * Send 10 000 small messages to InQueue to server 1
> * Then deploy MDB on server 2. MDB consumes messages from InQueue and resends to OutQueue
> * Wait until MDB processes all messages and check that 10 000 messages is in OutQueue
> Result:
> Sometimes happens that Artemis in server 1 crashes due to following exception:
> {code}
> 20:13:45,641 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) IO Error: Socket read
> timed out: java.sql.BatchUpdateException: IO Error: Socket read timed out
> at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
> at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
> at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
> at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
> at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> 20:13:45,652 WARN [org.apache.activemq.artemis.core.server] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) AMQ222010: Critical IO Error, shutting down the server. file=NULL, message=Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
> at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
> at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
> at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
> at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
> at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> 20:13:45,655 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) java.sql.SQLRecoverableException: Closed Connection
> {code}
> This thing can be very frustrating for customers who would rely on JDBC store in production. We should provide more robust solution and guide how to configure JDBC store to avoid this situation.
> This problem will arise also when moving to OpenShift which runs in cloud environment with unreliable and slow network to database like Amazon AWS, MS Azure or OpenStack.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (WFLY-10749) Artemis with JDBC store on slow or high latency database crashed on: Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
by Miroslav Novak (JIRA)
[ https://issues.jboss.org/browse/WFLY-10749?page=com.atlassian.jira.plugin... ]
Miroslav Novak updated WFLY-10749:
----------------------------------
Description:
Artemis configured with JDBC store is not robust if database is slow/under high load and when there is higher latency to database. Artemis has tendency to shutdown itself on "Critical IO Error" under really low load. It is visible when EAP QE moved Jenkins to Central CI where is higher latency to used Oracle12cR2 database. In following test scenario:
* Start 2 WF/EAP server in cluster. There is queue InQueue and OutQueue deployed on both of the servers.
* Send 10 000 small messages to InQueue to server 1
* Then deploy MDB on server 2. MDB consumes messages from InQueue and resends to OutQueue
* Wait until MDB processes all messages and check that 10 000 messages is in OutQueue
Result:
Sometimes happens that Artemis in server 1 crashes due to following exception:
{code}
20:13:45,641 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) IO Error: Socket read
timed out: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,652 WARN [org.apache.activemq.artemis.core.server] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) AMQ222010: Critical IO Error, shutting down the server. file=NULL, message=Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,655 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) java.sql.SQLRecoverableException: Closed Connection
{code}
This thing can be very frustrating for customers who would rely on JDBC store in production. We should provide more robust solution and guide how to configure JDBC store to avoid this situation.
This problem will arise also when moving to OpenShift which runs in cloud environment with unreliable and slow network to database like Amazon AWS, MS Azure or OpenStack.
was:
Artemis configured with JDBC store is not robust if database is slow/under high load and when there is higher latency to database. Artemis has tendency to shutdown itself on "Critical IO Error" under low load. It is visible when EAP QE moved Jenkins to Central CI where is higher latency to used Oracle12cR2 database. In following test scenario:
* Start 2 WF/EAP server in cluster. There is queue InQueue and OutQueue deployed on both of the servers.
* Send 10 000 small messages to InQueue to server 1
* Then deploy MDB on server 2. MDB consumes messages from InQueue and resends to OutQueue
* Wait until MDB processes all messages and check that 10 000 messages is in OutQueue
Result:
Sometimes happens that Artemis in server 1 crashes due to following exception:
{code}
20:13:45,641 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) IO Error: Socket read
timed out: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,652 WARN [org.apache.activemq.artemis.core.server] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) AMQ222010: Critical IO Error, shutting down the server. file=NULL, message=Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,655 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) java.sql.SQLRecoverableException: Closed Connection
{code}
This thing can be very frustrating for customers who would rely on JDBC store in production. We should provide more robust solution and guide how to configure JDBC store to avoid this situation.
This problem will arise also when moving to OpenShift which runs in cloud environment with unreliable and slow network to database like Amazon AWS, MS Azure or OpenStack.
> Artemis with JDBC store on slow or high latency database crashed on: Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-10749
> URL: https://issues.jboss.org/browse/WFLY-10749
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Affects Versions: 12.0.0.Final, 13.0.0.Final
> Reporter: Miroslav Novak
> Assignee: Martyn Taylor
> Priority: Critical
>
> Artemis configured with JDBC store is not robust if database is slow/under high load and when there is higher latency to database. Artemis has tendency to shutdown itself on "Critical IO Error" under really low load. It is visible when EAP QE moved Jenkins to Central CI where is higher latency to used Oracle12cR2 database. In following test scenario:
> * Start 2 WF/EAP server in cluster. There is queue InQueue and OutQueue deployed on both of the servers.
> * Send 10 000 small messages to InQueue to server 1
> * Then deploy MDB on server 2. MDB consumes messages from InQueue and resends to OutQueue
> * Wait until MDB processes all messages and check that 10 000 messages is in OutQueue
> Result:
> Sometimes happens that Artemis in server 1 crashes due to following exception:
> {code}
> 20:13:45,641 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) IO Error: Socket read
> timed out: java.sql.BatchUpdateException: IO Error: Socket read timed out
> at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
> at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
> at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
> at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
> at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> 20:13:45,652 WARN [org.apache.activemq.artemis.core.server] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) AMQ222010: Critical IO Error, shutting down the server. file=NULL, message=Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
> at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
> at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
> at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
> at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
> at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> 20:13:45,655 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) java.sql.SQLRecoverableException: Closed Connection
> {code}
> This thing can be very frustrating for customers who would rely on JDBC store in production. We should provide more robust solution and guide how to configure JDBC store to avoid this situation.
> This problem will arise also when moving to OpenShift which runs in cloud environment with unreliable and slow network to database like Amazon AWS, MS Azure or OpenStack.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (WFLY-10749) Artemis with JDBC store on slow or high latency database crashed on: Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
by Miroslav Novak (JIRA)
[ https://issues.jboss.org/browse/WFLY-10749?page=com.atlassian.jira.plugin... ]
Miroslav Novak updated WFLY-10749:
----------------------------------
Description:
Artemis configured with JDBC store is not robust if database is slow/under high load and when there is higher latency to database. Artemis has tendency to shutdown itself on "Critical IO Error" under low load. It is visible when EAP QE moved Jenkins to Central CI where is higher latency to used Oracle12cR2 database. In following test scenario:
* Start 2 WF/EAP server in cluster. There is queue InQueue and OutQueue deployed on both of the servers.
* Send 10 000 small messages to InQueue to server 1
* Then deploy MDB on server 2. MDB consumes messages from InQueue and resends to OutQueue
* Wait until MDB processes all messages and check that 10 000 messages is in OutQueue
Result:
Sometimes happens that Artemis in server 1 crashes due to following exception:
{code}
20:13:45,641 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) IO Error: Socket read
timed out: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,652 WARN [org.apache.activemq.artemis.core.server] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) AMQ222010: Critical IO Error, shutting down the server. file=NULL, message=Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,655 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) java.sql.SQLRecoverableException: Closed Connection
{code}
This thing can be very frustrating for customers who would rely on JDBC store in production. We should provide more robust solution and guide how to configure JDBC store to avoid this situation.
This problem will arise also when moving to OpenShift which runs in cloud environment with unreliable and slow network to database like Amazon AWS, MS Azure or OpenStack.
was:
Artemis configured with JDBC store is not robust if database is slow/under high load and when there is higher latency to database. Artemis has tendency to shutdown itself on "Critical IO Error" under low load. It is visible when EAP QE moved Jenkins to Central CI where is higher latency to used Oracle12cR2 database. In following test scenario:
* Start 2 WF/EAP server in cluster. There is queue InQueue and OutQueue deployed on both of the servers.
* Send 10 000 small messages to InQueue to server 1
* Then deploy MDB on server 2. MDB consumes messages from InQueue and resends to OutQueue
* Wait until MDB processes all messages and check that 10 000 messages is in OutQueue
Result:
Sometimes happens that Artemis in server 1 crashes due to following exception:
{code}
20:13:45,641 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) IO Error: Socket read
timed out: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,652 WARN [org.apache.activemq.artemis.core.server] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) AMQ222010: Critical IO Error, shutting down the server. file=NULL, message=Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,655 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) java.sql.SQLRecoverableException: Closed Connection
{code}
This thing can be very frustrating for customers who would rely on JDBC store in production. We should provide more robust solution and guide how to configure JDBC store to avoid this situation.
> Artemis with JDBC store on slow or high latency database crashed on: Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-10749
> URL: https://issues.jboss.org/browse/WFLY-10749
> Project: WildFly
> Issue Type: Bug
> Components: JMS
> Affects Versions: 12.0.0.Final, 13.0.0.Final
> Reporter: Miroslav Novak
> Assignee: Martyn Taylor
> Priority: Critical
>
> Artemis configured with JDBC store is not robust if database is slow/under high load and when there is higher latency to database. Artemis has tendency to shutdown itself on "Critical IO Error" under low load. It is visible when EAP QE moved Jenkins to Central CI where is higher latency to used Oracle12cR2 database. In following test scenario:
> * Start 2 WF/EAP server in cluster. There is queue InQueue and OutQueue deployed on both of the servers.
> * Send 10 000 small messages to InQueue to server 1
> * Then deploy MDB on server 2. MDB consumes messages from InQueue and resends to OutQueue
> * Wait until MDB processes all messages and check that 10 000 messages is in OutQueue
> Result:
> Sometimes happens that Artemis in server 1 crashes due to following exception:
> {code}
> 20:13:45,641 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) IO Error: Socket read
> timed out: java.sql.BatchUpdateException: IO Error: Socket read timed out
> at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
> at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
> at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
> at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
> at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> 20:13:45,652 WARN [org.apache.activemq.artemis.core.server] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) AMQ222010: Critical IO Error, shutting down the server. file=NULL, message=Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
> at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
> at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
> at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
> at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
> at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
> at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
> 20:13:45,655 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) java.sql.SQLRecoverableException: Closed Connection
> {code}
> This thing can be very frustrating for customers who would rely on JDBC store in production. We should provide more robust solution and guide how to configure JDBC store to avoid this situation.
> This problem will arise also when moving to OpenShift which runs in cloud environment with unreliable and slow network to database like Amazon AWS, MS Azure or OpenStack.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months
[JBoss JIRA] (WFLY-10749) Artemis with JDBC store on slow or high latency database crashed on: Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
by Miroslav Novak (JIRA)
Miroslav Novak created WFLY-10749:
-------------------------------------
Summary: Artemis with JDBC store on slow or high latency database crashed on: Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
Key: WFLY-10749
URL: https://issues.jboss.org/browse/WFLY-10749
Project: WildFly
Issue Type: Bug
Components: JMS
Affects Versions: 13.0.0.Final, 12.0.0.Final
Reporter: Miroslav Novak
Assignee: Martyn Taylor
Priority: Critical
Artemis configured with JDBC store is not robust if database is slow/under high load and when there is higher latency to database. Artemis has tendency to shutdown itself on "Critical IO Error" under low load. It is visible when EAP QE moved Jenkins to Central CI where is higher latency to used Oracle12cR2 database. In following test scenario:
* Start 2 WF/EAP server in cluster. There is queue InQueue and OutQueue deployed on both of the servers.
* Send 10 000 small messages to InQueue to server 1
* Then deploy MDB on server 2. MDB consumes messages from InQueue and resends to OutQueue
* Wait until MDB processes all messages and check that 10 000 messages is in OutQueue
Result:
Sometimes happens that Artemis in server 1 crashes due to following exception:
{code}
20:13:45,641 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) IO Error: Socket read
timed out: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,652 WARN [org.apache.activemq.artemis.core.server] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) AMQ222010: Critical IO Error, shutting down the server. file=NULL, message=Critical IO Error. Failed to process JDBC Record statements: java.sql.BatchUpdateException: IO Error: Socket read timed out
at oracle.jdbc.driver.OraclePreparedStatement.executeLargeBatch(OraclePreparedStatement.java:10032)
at oracle.jdbc.driver.T4CPreparedStatement.executeLargeBatch(T4CPreparedStatement.java:1364)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9839)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:234)
at org.jboss.jca.adapters.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:1180)
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl.sync(JDBCJournalImpl.java:228) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl$JDBCJournalSync.run(JDBCJournalImpl.java:991) [artemis-jdbc-store-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.core.server.ActiveMQScheduledComponent$2.run(ActiveMQScheduledComponent.java:284) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at org.apache.activemq.artemis.utils.OrderedExecutorFactory$OrderedExecutor$ExecutorTask.run(OrderedExecutorFactory.java:122) [artemis-commons-1.5.5.jbossorg-012.jar:1.5.5.jbossorg-012]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_171]
20:13:45,655 WARN [org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$3@4de066d8)) java.sql.SQLRecoverableException: Closed Connection
{code}
This thing can be very frustrating for customers who would rely on JDBC store in production. We should provide more robust solution and guide how to configure JDBC store to avoid this situation.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
7 years, 9 months