[JBoss JIRA] Created: (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
9:53 a.m.
Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public (Everyone can see)
Affects Versions: 2.5.4.SP2
Reporter: Artur Karpinski
Priority: Critical
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:37 p.m.
New subject: [JBoss JIRA] Updated: (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Ron Sigal updated JBREM-1297:
-----------------------------
Assignee: Ron Sigal
Fix Version/s: 2.5.4.SP4
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.5.4.SP2
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.5.4.SP4
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:22 p.m.
New subject: [JBoss JIRA] Updated: (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Ron Sigal updated JBREM-1297:
-----------------------------
Fix Version/s: 2.2.4.SP1
Affects Version/s: 2.2.4
2.5.4.SP3
(was: 2.5.4.SP2)
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:42 p.m.
New subject: [JBoss JIRA] Commented: (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Ron Sigal commented on JBREM-1297:
----------------------------------
Hi Artur,
You did indeed find a deadlock. Thank you!
JBREM-1242 didn't really introduce the deadlock, but rather it caused the sequence
Client.setupClientLease() -> MicroRemoteClientInvoker.establishLease()
to acquire Client.connectionValidatorLock before acquiring
MicroRemoteClientInvoker.clientLeaseLock, which aligned it with the sequence
Client.addConnectionListener() -> new ConnectionValidator() ->
MicroRemoteClientInvoker.getLeasePinger()
What you discovered was another sequence,
ConnectionValidator$WaitOnConnectionCheckTimerTask.run() ->
MicroRemoteClientInvoker.terminateLease() -> LeasePinger.stopPing() ->
Client.notifyListeners()
which was misaligned with the other two sequences. The fix I used is essentially your
suggestion, except I synchronized on Client.connectionValidatorLock in
ConnectionValidator$WaitOnConnectionCheckTimerTask.run() instead of
MicroRemoteClientInvoker.terminateLease().
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:44 p.m.
New subject: [JBoss JIRA] Commented: (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Ron Sigal commented on JBREM-1297:
----------------------------------
The fix is committed to branches 2.2 and 2.x. There is no specific unit test. Will watch
Jenkins for any anomalous behavior.
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
4:03 p.m.
New subject: [JBoss JIRA] Updated: (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Ron Sigal updated JBREM-1297:
-----------------------------
Attachment: jboss-remoting.jar
Attaching jboss-remoting.jar with fix for this issue.
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Attachments: jboss-remoting.jar
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12:26 p.m.
New subject: [JBoss JIRA] (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Justin Bertram commented on JBREM-1297:
---------------------------------------
Has this issue been resolved? If so, can you resolve the JIRA?
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Attachments: jboss-remoting.jar
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
1:14 p.m.
New subject: [JBoss JIRA] (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Ron Sigal commented on JBREM-1297:
----------------------------------
Hey Justin,
I've been getting some flaky runs on jenkins, which probably have nothing to do with
this issue, but I would like to see some reasonable results before I close the issue.
I'll run the test suites again today and see what happens.
=Ron
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Attachments: jboss-remoting.jar
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
10:54 a.m.
New subject: [JBoss JIRA] (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Justin Bertram commented on JBREM-1297:
---------------------------------------
Any update on this?
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Attachments: jboss-remoting.jar
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
11:20 a.m.
New subject: [JBoss JIRA] (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Ron Sigal commented on JBREM-1297:
----------------------------------
Hey Justin,
I'm getting some odd results in jenkins, which probably are completely unrelated, but
I want to look a little deeper. Sorry for the delay.
-Ron
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Attachments: jboss-remoting.jar
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
3:53 p.m.
New subject: [JBoss JIRA] (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Justin Bertram commented on JBREM-1297:
---------------------------------------
Any news in the last month or so?
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Attachments: jboss-remoting.jar
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:02 p.m.
New subject: [JBoss JIRA] (JBREM-1297) Deadlock between MicroRemoteClientInvoker.establishLease and Client.notifyListeners
[
https://issues.jboss.org/browse/JBREM-1297?page=com.atlassian.jira.plugin...
]
Ron Sigal resolved JBREM-1297.
------------------------------
Resolution: Done
Hi Justin,
Sorry about the delay. I ran the test suite again last week, and, despite some anomalies,
I don't see anything that would seem to be related to this issue, so I'm closing
it.
-Ron
Deadlock between MicroRemoteClientInvoker.establishLease and
Client.notifyListeners
-----------------------------------------------------------------------------------
Key: JBREM-1297
URL: https://issues.jboss.org/browse/JBREM-1297
Project: JBoss Remoting
Issue Type: Bug
Security Level: Public(Everyone can see)
Affects Versions: 2.2.4, 2.5.4.SP3
Reporter: Artur Karpinski
Assignee: Ron Sigal
Priority: Critical
Fix For: 2.2.4.SP1, 2.5.4.SP4
Attachments: jboss-remoting.jar
Deadlock is probably caused by changes introduced in this fix: JBREM-1242.
Thread A:
1. MicroRemoteClientInvoker.terminateLease, locks clientLeaseLock
2. Client.notifyListeners, locks connectionValidatorLock
Thread B:
1. Client.setupClientLease, locks connectionValidatorLock
2. MicroRemoteClientInvoker.establishLease, locks connectionValidatorLock
Please check stacktraces when this deadlock occurred:
{code}
"Timer-16" - Thread t@486453
java.lang.Thread.State: BLOCKED
at org.jboss.remoting.Client.notifyListeners(Client.java:1873)
- waiting to lock <747c39d3> (a java.lang.Object) owned by
"WorkManager(2)-241" t@6121
at org.jboss.remoting.LeasePinger.stopPing(LeasePinger.java:134)
at
org.jboss.remoting.MicroRemoteClientInvoker.terminateLease(MicroRemoteClientInvoker.java:434)
- locked <751714e> (a java.lang.Object)
at
org.jboss.remoting.ConnectionValidator$WaitOnConnectionCheckTimerTask.run(ConnectionValidator.java:1081)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
{code}
and
{code}
"WorkManager(2)-241" - Thread t@6121
java.lang.Thread.State: BLOCKED
at
org.jboss.remoting.MicroRemoteClientInvoker.establishLease(MicroRemoteClientInvoker.java:507)
- waiting to lock <751714e> (a java.lang.Object) owned by "Timer-16"
t@486453
at org.jboss.remoting.Client.setupClientLease(Client.java:2056)
- locked <747c39d3> (a java.lang.Object)
at org.jboss.remoting.Client.connect(Client.java:1918)
at org.jboss.remoting.Client.connect(Client.java:737)
{code}
One way to fix this would be to add lock on connectionValidatorLock in
MicroRemoteClientInvoker.terminateLease. I'm trying to analyse code to check if this
fix will not introduce another deadlock problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
4493
days inactive
4675
days old
jboss-remoting-issues@lists.jboss.org
11 comments
5 participants
participants (5)
-
Artur Karpinski (JIRA) -
Justin Bertram (Commented) (JIRA)
-
Ron Sigal (Commented) (JIRA)
-
Ron Sigal (JIRA)
-
Ron Sigal (Resolved) (JIRA)