[jboss-user] [jBPM] - Quartz Deadlocks

[aaron ehrensberger]

aaron ehrensberger [http://community.jboss.org/people/aehrensberger] created the discussion

"Quartz Deadlocks"

To view the discussion, visit: http://community.jboss.org/message/533394#533394

--------------------------------------------------------------
We are using Quartz with  Spring and our configuration is throwing deadlocks when quartz has more than 1  thread configured. I'm starting to believe that it's because we don't have our  quartz configured correctly with Spring, but I can't find enough documentation  on how to configure the two to play nicely.  I'm also starting to believe that Quartz is not going to be our solution, unfortunately, because of this issue.

We are running tests under heavy load, starting as many as 1000 jobs in a matter of minutes, all scheduled to be picked up by 50 quartz threads after 10 second delays.

We are running on both  Windows and Linux environments - pointing at MSSQL and Oracle DBs. With both OS,  using either DB, we can throw the following deadlock errors...

2010-03-17 18:52:31,737 [] [] ERROR [DocFinityScheduler_Worker-42]  core.ErrorLogger core.ErrorLogger (QuartzScheduler.java:2185) - An error occured  while marking executed job complete. job= 'BPM.6e41a6567f0000020100362a51dc7a50'

org.quartz.JobPersistenceException: Couldn't remove trigger: Transaction  (Process ID 87) was deadlocked on lock resources with another process and has  been chosen as the deadlock victim. Rerun the transaction. [See nested  exception: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process  ID 87) was deadlocked on lock resources with another process and has been chosen  as the deadlock victim. Rerun the transaction.] 
at  org.quartz.impl.jdbcjobstore.JobStoreSupport.removeTrigger(JobStoreSupport.java:1469)at  org.quartz.impl.jdbcjobstore.JobStoreSupport.triggeredJobComplete(JobStoreSupport.java:2978)at  org.quartz.impl.jdbcjobstore.JobStoreSupport$39.execute(JobStoreSupport.java:2962)  at  org.quartz.impl.jdbcjobstore.JobStoreSupport$41.execute(JobStoreSupport.java:3713)at  org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3747)  
at  org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3709)at  org.quartz.impl.jdbcjobstore.JobStoreSupport.triggeredJobComplete(JobStoreSupport.java:2958)at  org.quartz.core.QuartzScheduler.notifyJobStoreJobComplete(QuartzScheduler.java:1727)at  org.quartz.core.JobRunShell.run(JobRunShell.java:273) 
at  org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:534)

Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction  (Process ID 87) was deadlocked on lock resources with another process and has  been chosen as the deadlock victim. Rerun the transaction. 
at  com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(Unknown  Source at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(Unknown  Source) at  com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(Unknown  Source) at  com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(Unknown  Source) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(Unknown Source) at  com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(Unknown Source)  at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(Unknown  Source)at  com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(Unknown Source)  at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(Unknown  Source)at  com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:105)  at  org.quartz.impl.jdbcjobstore.StdJDBCDelegate.deleteSimpleTrigger(StdJDBCDelegate.java:1820)  at  org.quartz.impl.jdbcjobstore.JobStoreSupport.deleteTriggerAndChildren(JobStoreSupport.java:1345  at  org.quartz.impl.jdbcjobstore.JobStoreSupport.removeTrigger(JobStoreSupport.java:1453  ... 9 more


I have attached as many files as I could think of that relate to the issue.  Can anybody help me out here?   Do I have it  configured incorrectly to not use a JobStoreCMT versus a JobStoreTX? Do I have  it configured incorrectly in terms of threads or something else?

I'm  really at a loss. We've been scouring the web for this stuff for documentation  for two days and are still scratching our heads.

--------------------------------------------------------------

Reply to this message by going to Community
[http://community.jboss.org/message/533394#533394]

Start a new discussion in jBPM at Community
[http://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2034]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/jboss-user/attachments/20100322/b537616c/attachment.html