[JBoss JIRA] (DROOLS-2174) Deadlock between AbstractWorkingMemory and DefaultAgenda
by Mario Fusco (JIRA)
[ https://issues.jboss.org/browse/DROOLS-2174?page=com.atlassian.jira.plugi... ]
Mario Fusco commented on DROOLS-2174:
-------------------------------------
Do you have a reproducer for this? Otherwise I don't see how I could investigate and eventually fix this problem.
> Deadlock between AbstractWorkingMemory and DefaultAgenda
> --------------------------------------------------------
>
> Key: DROOLS-2174
> URL: https://issues.jboss.org/browse/DROOLS-2174
> Project: Drools
> Issue Type: Bug
> Affects Versions: 5.5.0.Final
> Reporter: Umang Sharma
> Assignee: Mario Fusco
> Fix For: 6.0.0.CR2
>
>
> I am getting a similar deadlock as the one in the cloned issue. We are using following dependencies:
> <dependency>
> <groupId>org.drools</groupId>
> <artifactId>drools-templates</artifactId>
> <version>$
> {runtime.version}</version>
> </dependency>
> <dependency>
> <groupId>org.kie</groupId>
> <artifactId>kie-api</artifactId>
> <version>${runtime.version}
> </version>
> </dependency>
> <dependency>
> <groupId>org.drools</groupId>
> <artifactId>drools-core</artifactId>
> <version>$
> {runtime.version}</version>
> </dependency>
> <dependency>
> <groupId>org.drools</groupId>
> <artifactId>drools-compiler</artifactId>
> <version>${runtime.version}
> </version>
> </dependency>
> runtime.version = 6.5.0-Final
> I have around 50 off threads stuck in a deadlock. 49 of them are having following stack trace:
> "THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
> at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
> And one of them is stuck in the following stack trace:
> "THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
> at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
> Please help us out in fixing this. Its a production system serving millions of users.
> PS: This is was reproduced under a medium load where we create 50 stateful sessions in parallel and also some rule were also getting added in the KieContainer at the same time.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 7 months
[JBoss JIRA] (DROOLS-2174) Deadlock between AbstractWorkingMemory and DefaultAgenda
by Umang Sharma (JIRA)
[ https://issues.jboss.org/browse/DROOLS-2174?page=com.atlassian.jira.plugi... ]
Umang Sharma updated DROOLS-2174:
---------------------------------
Description:
I am getting a similar deadlock as the one in the cloned issue. We are using following dependencies:
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-templates</artifactId>
<version>$
{runtime.version}</version>
</dependency>
<dependency>
<groupId>org.kie</groupId>
<artifactId>kie-api</artifactId>
<version>${runtime.version}
</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-core</artifactId>
<version>$
{runtime.version}</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-compiler</artifactId>
<version>${runtime.version}
</version>
</dependency>
runtime.version = 6.5.0-Final
I have around 50 off threads stuck in a deadlock. 49 of them are having following stack trace:
"THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
And one of them is stuck in the following stack trace:
"THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
Please help us out in fixing this. Its a production system serving millions of users.
PS: This is was reproduced under a medium load where we create 50 stateful sessions in parallel and also some rule were also getting added in the KieContainer at the same time.
was:
I experienced a deadlock when running in STREAM mode and using negative patterns with temporal constraints.
When the rule gets fired by the installed timer job and at the same time another thread is firing rules, a deadlock can occur:
1. [rule firing thread] StatefulKnowledgeSessionImpl.execute(StatefulKnowledgeSessionImpl.java:820)
acquires lock in AbstractWorkingMemory
2. [timer-thread] DefaultAgenda.fireActivation()
call of synchronized method
3. [rule firing thread] DefaultAgenda.fireActivation()
synchronized method, wait for other thread to exit synchornized method
4. [timer-thread] AbstractWorkingMemory.getGlobal()
tries to access a global, tries to acquire lock in AbstractWorkingMemory which is held by the other thread
--> Deadlock
Excerpt from thread dump:
"http-thread-pool-8082(374)":
at org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1264)
- waiting to lock <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
at org.drools.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:1221)
at org.drools.common.DefaultAgenda.fireAllRules(DefaultAgenda.java:1456)
at org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:710)
at org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:679)
at org.drools.impl.StatefulKnowledgeSessionImpl.fireAllRules(StatefulKnowledgeSessionImpl.java:234)
at org.drools.command.runtime.rule.FireAllRulesCommand.execute(FireAllRulesCommand.java:102)
at org.drools.command.runtime.rule.FireAllRulesCommand.execute(FireAllRulesCommand.java:32)
at org.drools.command.runtime.BatchExecutionCommandImpl.execute(BatchExecutionCommandImpl.java:155)
at org.drools.command.runtime.BatchExecutionCommandImpl.execute(BatchExecutionCommandImpl.java:76)
at org.drools.impl.StatefulKnowledgeSessionImpl.execute(StatefulKnowledgeSessionImpl.java:821)
"pool-4251-thread-1":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for 0x00007f503fc71a20> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
at org.drools.common.AbstractWorkingMemory.getGlobal(AbstractWorkingMemory.java:634)
at myrules.Rule_ruleWithTimeout_ece4a3c9fc2741a4ab97a92b77611203DefaultConsequenceInvokerGenerated.evaluate(Unknown Source)
at org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1287)
- locked <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
at org.drools.common.DefaultAgenda.fireTimedActivation(DefaultAgenda.java:1344)
- locked <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
at org.drools.common.Scheduler$ActivationTimerJob.execute(Scheduler.java:83)
> Deadlock between AbstractWorkingMemory and DefaultAgenda
> --------------------------------------------------------
>
> Key: DROOLS-2174
> URL: https://issues.jboss.org/browse/DROOLS-2174
> Project: Drools
> Issue Type: Bug
> Affects Versions: 5.5.0.Final
> Reporter: Umang Sharma
> Assignee: Mario Fusco
> Fix For: 6.0.0.CR2
>
>
> I am getting a similar deadlock as the one in the cloned issue. We are using following dependencies:
> <dependency>
> <groupId>org.drools</groupId>
> <artifactId>drools-templates</artifactId>
> <version>$
> {runtime.version}</version>
> </dependency>
> <dependency>
> <groupId>org.kie</groupId>
> <artifactId>kie-api</artifactId>
> <version>${runtime.version}
> </version>
> </dependency>
> <dependency>
> <groupId>org.drools</groupId>
> <artifactId>drools-core</artifactId>
> <version>$
> {runtime.version}</version>
> </dependency>
> <dependency>
> <groupId>org.drools</groupId>
> <artifactId>drools-compiler</artifactId>
> <version>${runtime.version}
> </version>
> </dependency>
> runtime.version = 6.5.0-Final
> I have around 50 off threads stuck in a deadlock. 49 of them are having following stack trace:
> "THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
> at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
> And one of them is stuck in the following stack trace:
> "THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
> at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
> at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
> at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
> Please help us out in fixing this. Its a production system serving millions of users.
> PS: This is was reproduced under a medium load where we create 50 stateful sessions in parallel and also some rule were also getting added in the KieContainer at the same time.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 7 months
[JBoss JIRA] (DROOLS-2174) Deadlock between AbstractWorkingMemory and DefaultAgenda
by Umang Sharma (JIRA)
Umang Sharma created DROOLS-2174:
------------------------------------
Summary: Deadlock between AbstractWorkingMemory and DefaultAgenda
Key: DROOLS-2174
URL: https://issues.jboss.org/browse/DROOLS-2174
Project: Drools
Issue Type: Bug
Affects Versions: 5.5.0.Final
Reporter: Umang Sharma
Assignee: Mario Fusco
Fix For: 6.0.0.CR2
I experienced a deadlock when running in STREAM mode and using negative patterns with temporal constraints.
When the rule gets fired by the installed timer job and at the same time another thread is firing rules, a deadlock can occur:
1. [rule firing thread] StatefulKnowledgeSessionImpl.execute(StatefulKnowledgeSessionImpl.java:820)
acquires lock in AbstractWorkingMemory
2. [timer-thread] DefaultAgenda.fireActivation()
call of synchronized method
3. [rule firing thread] DefaultAgenda.fireActivation()
synchronized method, wait for other thread to exit synchornized method
4. [timer-thread] AbstractWorkingMemory.getGlobal()
tries to access a global, tries to acquire lock in AbstractWorkingMemory which is held by the other thread
--> Deadlock
Excerpt from thread dump:
"http-thread-pool-8082(374)":
at org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1264)
- waiting to lock <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
at org.drools.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:1221)
at org.drools.common.DefaultAgenda.fireAllRules(DefaultAgenda.java:1456)
at org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:710)
at org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:679)
at org.drools.impl.StatefulKnowledgeSessionImpl.fireAllRules(StatefulKnowledgeSessionImpl.java:234)
at org.drools.command.runtime.rule.FireAllRulesCommand.execute(FireAllRulesCommand.java:102)
at org.drools.command.runtime.rule.FireAllRulesCommand.execute(FireAllRulesCommand.java:32)
at org.drools.command.runtime.BatchExecutionCommandImpl.execute(BatchExecutionCommandImpl.java:155)
at org.drools.command.runtime.BatchExecutionCommandImpl.execute(BatchExecutionCommandImpl.java:76)
at org.drools.impl.StatefulKnowledgeSessionImpl.execute(StatefulKnowledgeSessionImpl.java:821)
"pool-4251-thread-1":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for 0x00007f503fc71a20> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
at org.drools.common.AbstractWorkingMemory.getGlobal(AbstractWorkingMemory.java:634)
at myrules.Rule_ruleWithTimeout_ece4a3c9fc2741a4ab97a92b77611203DefaultConsequenceInvokerGenerated.evaluate(Unknown Source)
at org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1287)
- locked <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
at org.drools.common.DefaultAgenda.fireTimedActivation(DefaultAgenda.java:1344)
- locked <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
at org.drools.common.Scheduler$ActivationTimerJob.execute(Scheduler.java:83)
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 7 months
[JBoss JIRA] (DROOLS-2174) Deadlock between AbstractWorkingMemory and DefaultAgenda
by Umang Sharma (JIRA)
[ https://issues.jboss.org/browse/DROOLS-2174?page=com.atlassian.jira.plugi... ]
Umang Sharma commented on DROOLS-2174:
--------------------------------------
I would like someone to reopen this issue. I am getting a similar deadlock. Using below dependencies:
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-templates</artifactId>
<version>$
{runtime.version}</version>
</dependency>
<dependency>
<groupId>org.kie</groupId>
<artifactId>kie-api</artifactId>
<version>${runtime.version}
</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-core</artifactId>
<version>$
{runtime.version}</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-compiler</artifactId>
<version>${runtime.version}
</version>
</dependency>
runtime.version = 6.5.0-Final
I have around 50 off threads stuck in a deadlock. 49 of them are having following stack trace:
"THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
And one of them is stuck in the following stack trace:
"THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
Please help us out in fixing this. Its a production system serving millions of users.
PS: This is was reproduced under a medium load where we create 50 stateful sessions in parallel and also some rule were also getting added in the KieContainer at the same time.
> Deadlock between AbstractWorkingMemory and DefaultAgenda
> --------------------------------------------------------
>
> Key: DROOLS-2174
> URL: https://issues.jboss.org/browse/DROOLS-2174
> Project: Drools
> Issue Type: Bug
> Affects Versions: 5.5.0.Final
> Reporter: Umang Sharma
> Assignee: Mario Fusco
> Fix For: 6.0.0.CR2
>
>
> I experienced a deadlock when running in STREAM mode and using negative patterns with temporal constraints.
> When the rule gets fired by the installed timer job and at the same time another thread is firing rules, a deadlock can occur:
> 1. [rule firing thread] StatefulKnowledgeSessionImpl.execute(StatefulKnowledgeSessionImpl.java:820)
> acquires lock in AbstractWorkingMemory
> 2. [timer-thread] DefaultAgenda.fireActivation()
> call of synchronized method
> 3. [rule firing thread] DefaultAgenda.fireActivation()
> synchronized method, wait for other thread to exit synchornized method
> 4. [timer-thread] AbstractWorkingMemory.getGlobal()
> tries to access a global, tries to acquire lock in AbstractWorkingMemory which is held by the other thread
> --> Deadlock
> Excerpt from thread dump:
> "http-thread-pool-8082(374)":
> at org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1264)
> - waiting to lock <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
> at org.drools.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:1221)
> at org.drools.common.DefaultAgenda.fireAllRules(DefaultAgenda.java:1456)
> at org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:710)
> at org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:679)
> at org.drools.impl.StatefulKnowledgeSessionImpl.fireAllRules(StatefulKnowledgeSessionImpl.java:234)
> at org.drools.command.runtime.rule.FireAllRulesCommand.execute(FireAllRulesCommand.java:102)
> at org.drools.command.runtime.rule.FireAllRulesCommand.execute(FireAllRulesCommand.java:32)
> at org.drools.command.runtime.BatchExecutionCommandImpl.execute(BatchExecutionCommandImpl.java:155)
> at org.drools.command.runtime.BatchExecutionCommandImpl.execute(BatchExecutionCommandImpl.java:76)
> at org.drools.impl.StatefulKnowledgeSessionImpl.execute(StatefulKnowledgeSessionImpl.java:821)
> "pool-4251-thread-1":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for 0x00007f503fc71a20> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
> at org.drools.common.AbstractWorkingMemory.getGlobal(AbstractWorkingMemory.java:634)
> at myrules.Rule_ruleWithTimeout_ece4a3c9fc2741a4ab97a92b77611203DefaultConsequenceInvokerGenerated.evaluate(Unknown Source)
> at org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1287)
> - locked <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
> at org.drools.common.DefaultAgenda.fireTimedActivation(DefaultAgenda.java:1344)
> - locked <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
> at org.drools.common.Scheduler$ActivationTimerJob.execute(Scheduler.java:83)
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 7 months
[JBoss JIRA] (DROOLS-206) Deadlock between AbstractWorkingMemory and DefaultAgenda
by Umang Sharma (JIRA)
[ https://issues.jboss.org/browse/DROOLS-206?page=com.atlassian.jira.plugin... ]
Umang Sharma commented on DROOLS-206:
-------------------------------------
I would like someone to reopen this issue. I am getting a similar deadlock. Using below dependencies:
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-templates</artifactId>
<version>${runtime.version}</version>
</dependency>
<dependency>
<groupId>org.kie</groupId>
<artifactId>kie-api</artifactId>
<version>${runtime.version}</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-core</artifactId>
<version>${runtime.version}</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-compiler</artifactId>
<version>${runtime.version}</version>
</dependency>
runtime.version = 6.5.0-Final
I have around 50 off threads stuck in a deadlock. 49 of them are having following stack trace:
"THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
And one of them is stuck in the following stack trace:
"THREAD-4" #124 prio=5 os_prio=0 tid=0x00000000015fc800 nid=0x3a5d waiting on condition [0x00007fc773e7b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000074d189818> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
at org.drools.core.impl.KnowledgeBaseImpl.readLock(KnowledgeBaseImpl.java:703)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1434)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulSession(KnowledgeBaseImpl.java:1422)
at org.drools.core.impl.KnowledgeBaseImpl.newStatefulKnowledgeSession(KnowledgeBaseImpl.java:308)
at org.drools.core.impl.KnowledgeBaseImpl.newKieSession(KnowledgeBaseImpl.java:385)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:693)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:629)
at org.drools.compiler.kie.builder.impl.KieContainerImpl.newKieSession(KieContainerImpl.java:611)
Please help us out in fixing this. Its a production system serving millions of users.
PS: This is was reproduced under a medium load where we create 50 stateful sessions in parallel and also some rule were also getting added in the KieContainer at the same time.
> Deadlock between AbstractWorkingMemory and DefaultAgenda
> --------------------------------------------------------
>
> Key: DROOLS-206
> URL: https://issues.jboss.org/browse/DROOLS-206
> Project: Drools
> Issue Type: Bug
> Affects Versions: 5.5.0.Final
> Reporter: Winfried Umbrath
> Assignee: Mario Fusco
> Fix For: 5.5.1.Final, 6.0.0.CR2
>
> Attachments: jbrules-3700-test.tgz
>
>
> I experienced a deadlock when running in STREAM mode and using negative patterns with temporal constraints.
> When the rule gets fired by the installed timer job and at the same time another thread is firing rules, a deadlock can occur:
> 1. [rule firing thread] StatefulKnowledgeSessionImpl.execute(StatefulKnowledgeSessionImpl.java:820)
> acquires lock in AbstractWorkingMemory
> 2. [timer-thread] DefaultAgenda.fireActivation()
> call of synchronized method
> 3. [rule firing thread] DefaultAgenda.fireActivation()
> synchronized method, wait for other thread to exit synchornized method
> 4. [timer-thread] AbstractWorkingMemory.getGlobal()
> tries to access a global, tries to acquire lock in AbstractWorkingMemory which is held by the other thread
> --> Deadlock
> Excerpt from thread dump:
> "http-thread-pool-8082(374)":
> at org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1264)
> - waiting to lock <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
> at org.drools.common.DefaultAgenda.fireNextItem(DefaultAgenda.java:1221)
> at org.drools.common.DefaultAgenda.fireAllRules(DefaultAgenda.java:1456)
> at org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:710)
> at org.drools.common.AbstractWorkingMemory.fireAllRules(AbstractWorkingMemory.java:679)
> at org.drools.impl.StatefulKnowledgeSessionImpl.fireAllRules(StatefulKnowledgeSessionImpl.java:234)
> at org.drools.command.runtime.rule.FireAllRulesCommand.execute(FireAllRulesCommand.java:102)
> at org.drools.command.runtime.rule.FireAllRulesCommand.execute(FireAllRulesCommand.java:32)
> at org.drools.command.runtime.BatchExecutionCommandImpl.execute(BatchExecutionCommandImpl.java:155)
> at org.drools.command.runtime.BatchExecutionCommandImpl.execute(BatchExecutionCommandImpl.java:76)
> at org.drools.impl.StatefulKnowledgeSessionImpl.execute(StatefulKnowledgeSessionImpl.java:821)
> "pool-4251-thread-1":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for 0x00007f503fc71a20> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
> at org.drools.common.AbstractWorkingMemory.getGlobal(AbstractWorkingMemory.java:634)
> at myrules.Rule_ruleWithTimeout_ece4a3c9fc2741a4ab97a92b77611203DefaultConsequenceInvokerGenerated.evaluate(Unknown Source)
> at org.drools.common.DefaultAgenda.fireActivation(DefaultAgenda.java:1287)
> - locked <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
> at org.drools.common.DefaultAgenda.fireTimedActivation(DefaultAgenda.java:1344)
> - locked <0x00007f50407e4950> (a org.drools.common.DefaultAgenda)
> at org.drools.common.Scheduler$ActivationTimerJob.execute(Scheduler.java:83)
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 7 months
[JBoss JIRA] (WFLY-9610) Start of a BatchJob is called, but BatchJob is seems no started. Absent entries in DB tables step_execution, job_execution
by Cheng Fang (JIRA)
[ https://issues.jboss.org/browse/WFLY-9610?page=com.atlassian.jira.plugin.... ]
Cheng Fang commented on WFLY-9610:
----------------------------------
starting a job execution is an async operation, so it is not guaranteed to run right away, but eventually the job execution should be running and records (job instance, job execution, step execution, and maybe partition execution) will be added to the job repository.
endtime and exitStatus are not present in table probably because the job execution has not finished yet so these data are not available.
Have you tried running it on more recent versions of WildFly (version 10.1, or 11)?
> Start of a BatchJob is called, but BatchJob is seems no started. Absent entries in DB tables step_execution, job_execution
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-9610
> URL: https://issues.jboss.org/browse/WFLY-9610
> Project: WildFly
> Issue Type: Bug
> Components: Batch, Clustering
> Affects Versions: 9.0.1.Final
> Environment: Cluster, standalone-full-ha
> Reporter: Serg Pol
> Assignee: Cheng Fang
>
> Start of a BatchJob is called and record/entry is absent sometimes in DB table "step_execution" as well as Endtime and Exitstatus in the table job_execution (there is just info about start of BatchJob).
> There are no any error nessages.
> BatchJob is not started in this case according Log.
> Any idea? Thanks
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 7 months
[JBoss JIRA] (AG-46) Fix intermitent failing tests
by Luis Barreiro (JIRA)
Luis Barreiro created AG-46:
-------------------------------
Summary: Fix intermitent failing tests
Key: AG-46
URL: https://issues.jboss.org/browse/AG-46
Project: Agroal
Issue Type: Bug
Components: test
Affects Versions: 0.3
Reporter: Luis Barreiro
Assignee: Luis Barreiro
Priority: Minor
Fix For: 0.4
Increase the timeouts in BasicTests
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 7 months
[JBoss JIRA] (AG-45) Don't set the transaction isolation level
by Luis Barreiro (JIRA)
Luis Barreiro created AG-45:
-------------------------------
Summary: Don't set the transaction isolation level
Key: AG-45
URL: https://issues.jboss.org/browse/AG-45
Project: Agroal
Issue Type: Bug
Components: api
Affects Versions: 0.3
Reporter: Luis Barreiro
Assignee: Luis Barreiro
Fix For: 0.4
By default, the transaction isolation level should not be set. The JDBC driver will pick the default value.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
8 years, 7 months