[
https://jira.jboss.org/browse/JBRULES-2651?page=com.atlassian.jira.plugin...
]
Norman Chan updated JBRULES-2651:
---------------------------------
Description:
I found this issue when my server stopped processing new requests and I did a thread dump.
In examining the dump, I found that all of the processing threads, save two, were
blocking while trying to acquire the lock in NamedEntryPoint.insert. Both of the other
two threads appeared to be infinitely looping in the NamedEntryPoint.insert method. Here
are snippets of their stack traces:
ActiveMQ Session Task" prio=10 tid=0x00002aab0003b000 nid=0x7b98 runnable
[0x000000004c086000..0x000000004c087c90] java.lang.Thread.State: RUNNABLE
at org.drools.util.ObjectHashMap.remove(ObjectHashMap.java:121) at
org.drools.common.SingleThreadedObjectStore.removeHandle(SingleThreadedObjectStore.java:150)
at org.drools.common.NamedEntryPoint.retract(NamedEntryPoint.java:296)
at org.drools.common.NamedEntryPoint.retract(NamedEntryPoint.java:245) at
org.drools.reteoo.ReteooWorkingMemory$WorkingMemoryReteExpireAction.execute(ReteooWorkingMemory.java:350)
at
org.drools.common.AbstractWorkingMemory.executeQueuedActions(AbstractWorkingMemory.java:1488)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:158)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:122) at
org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:80) at
org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:28) at
ActiveMQ Session Task" prio=10 tid=0x000000005a35cc00 nid=0xdf6 runnable
[0x000000004a268000..0x000000004a269a90] java.lang.Thread.State: RUNNABLE
at org.drools.util.AbstractHashTable.resize(AbstractHashTable.java:115) at
org.drools.util.ObjectHashMap.put(ObjectHashMap.java:78) at
org.drools.common.SingleThreadedObjectStore.addHandle(SingleThreadedObjectStore.java:136)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:113)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:80) at
org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:28) at
So it seems like while the first thread is holding the lock and attempting to remove an
object handle from the object store in NamedEntryPoint, the other thread is trying to
resize that same object store in response to an addHandle call that puts it over the
threshold. I haven't worked out exactly how these concurrent accesses to the same
object store by two different threads causes an infinite loop in both threads, but it
seems like the call to SingleThreadedObjectStore.addHandle should be preceded by acquiring
the lock.
was:
I found issue which I believe might point to a concurrency issue. My server stopped
processing new requests, so I did a thread dump. In examining the dump, I found that all
of the processing threads, save two, were blocking while trying to acquire the lock in
NamedEntryPoint.insert. Both of the other two threads appeared to be infinitely looping
in the NamedEntryPoint.insert method. Here are snippets of their stack traces:
ActiveMQ Session Task" prio=10 tid=0x00002aab0003b000 nid=0x7b98 runnable
[0x000000004c086000..0x000000004c087c90] java.lang.Thread.State: RUNNABLE
at org.drools.util.ObjectHashMap.remove(ObjectHashMap.java:121) at
org.drools.common.SingleThreadedObjectStore.removeHandle(SingleThreadedObjectStore.java:150)
at org.drools.common.NamedEntryPoint.retract(NamedEntryPoint.java:296)
at org.drools.common.NamedEntryPoint.retract(NamedEntryPoint.java:245) at
org.drools.reteoo.ReteooWorkingMemory$WorkingMemoryReteExpireAction.execute(ReteooWorkingMemory.java:350)
at
org.drools.common.AbstractWorkingMemory.executeQueuedActions(AbstractWorkingMemory.java:1488)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:158)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:122) at
org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:80) at
org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:28) at
ActiveMQ Session Task" prio=10 tid=0x000000005a35cc00 nid=0xdf6 runnable
[0x000000004a268000..0x000000004a269a90] java.lang.Thread.State: RUNNABLE
at org.drools.util.AbstractHashTable.resize(AbstractHashTable.java:115) at
org.drools.util.ObjectHashMap.put(ObjectHashMap.java:78) at
org.drools.common.SingleThreadedObjectStore.addHandle(SingleThreadedObjectStore.java:136)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:113)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:80) at
org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:28) at
So it seems like while the first thread is holding the lock and attempting to remove an
object handle from the object store in NamedEntryPoint, the other thread is trying to
resize that same object store in response to an addHandle call that puts it over the
threshold. I haven't worked out exactly how these concurrent accesses to the same
object store by two different threads causes an infinite loop in both threads, but it
seems like the call to SingleThreadedObjectStore.addHandle should be preceded by acquiring
the lock.
Concurrency issue in NamedEntryPoint.insert leads to occasional
hanging in Drools
---------------------------------------------------------------------------------
Key: JBRULES-2651
URL:
https://jira.jboss.org/browse/JBRULES-2651
Project: Drools
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: drools-core (expert)
Affects Versions: 5.0.1.FINAL
Reporter: Norman Chan
Assignee: Mark Proctor
I found this issue when my server stopped processing new requests and I did a thread
dump. In examining the dump, I found that all of the processing threads, save two, were
blocking while trying to acquire the lock in NamedEntryPoint.insert. Both of the other
two threads appeared to be infinitely looping in the NamedEntryPoint.insert method. Here
are snippets of their stack traces:
ActiveMQ Session Task" prio=10 tid=0x00002aab0003b000 nid=0x7b98 runnable
[0x000000004c086000..0x000000004c087c90] java.lang.Thread.State: RUNNABLE
at org.drools.util.ObjectHashMap.remove(ObjectHashMap.java:121) at
org.drools.common.SingleThreadedObjectStore.removeHandle(SingleThreadedObjectStore.java:150)
at org.drools.common.NamedEntryPoint.retract(NamedEntryPoint.java:296)
at org.drools.common.NamedEntryPoint.retract(NamedEntryPoint.java:245) at
org.drools.reteoo.ReteooWorkingMemory$WorkingMemoryReteExpireAction.execute(ReteooWorkingMemory.java:350)
at
org.drools.common.AbstractWorkingMemory.executeQueuedActions(AbstractWorkingMemory.java:1488)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:158)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:122) at
org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:80) at
org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:28) at
ActiveMQ Session Task" prio=10 tid=0x000000005a35cc00 nid=0xdf6 runnable
[0x000000004a268000..0x000000004a269a90] java.lang.Thread.State: RUNNABLE
at org.drools.util.AbstractHashTable.resize(AbstractHashTable.java:115) at
org.drools.util.ObjectHashMap.put(ObjectHashMap.java:78) at
org.drools.common.SingleThreadedObjectStore.addHandle(SingleThreadedObjectStore.java:136)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:113)
at org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:80) at
org.drools.common.NamedEntryPoint.insert(NamedEntryPoint.java:28) at
So it seems like while the first thread is holding the lock and attempting to remove an
object handle from the object store in NamedEntryPoint, the other thread is trying to
resize that same object store in response to an addHandle call that puts it over the
threshold. I haven't worked out exactly how these concurrent accesses to the same
object store by two different threads causes an infinite loop in both threads, but it
seems like the call to SingleThreadedObjectStore.addHandle should be preceded by acquiring
the lock.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira