[jboss-jira] [JBoss JIRA] Closed: (JGRP-614) FC: deadlock with synchronous RPCs and exhausted credits

[Bela Ban (JIRA)]

     [ http://jira.jboss.com/jira/browse/JGRP-614?page=all ]

Bela Ban closed JGRP-614.
-------------------------

    Resolution: Won't Fix

After some discussion, we decided not to fix this, as (a) we haven't really seen many of these cases and http://jira.jboss.com/jira/browse/JGRP-465 fixes it to a certain degree and (b) it would be too much of a change to backport parts of the concurrent stack to a patch release (SP5). On top, it would also require adding a 'flag' instance var to Message, thereby violating binary backward compatibility in patch releases.

Workaround: remove FC (as recommended for synchronous RPCs anyway across documentation), or upgrade to JGroups 2.5.

> FC: deadlock with synchronous RPCs and exhausted credits
> --------------------------------------------------------
>
>                 Key: JGRP-614
>                 URL: http://jira.jboss.com/jira/browse/JGRP-614
>             Project: JGroups
>          Issue Type: Feature Request
>            Reporter: Bela Ban
>         Assigned To: Bela Ban
>             Fix For: 2.4.1 SP5
>
>
> # Members A and B
> # A makes a *synchronous* clustered RPC
> # B handles the RPC by placing the message into a queue where it is processed by the one and only incoming handler thread.
> # The handler thread dequeues the message and passes it up the stack
> # In NAKACK, a lock(A) is acquired and the message is passed up
> # The app receives the message, invokes the method and - since it is synchronous - passes the response down
> # In FC.down(), we happen to run out of credits and block
> # FC.down() periodically requests credits from A.
> # A receives B's credit request and sends new credits to B
> # B receives the credits and places them into the incoming handler's queue
> # HOWEVER, B will never get the credits because its incoming handler thread is still stuck in FC.down() and never has a chance to process that message
> We currently suggest not to use FC with synchronous RPCs on JGroups 2.4.1 (note that 2.5 and higher doesn't have this issue).
> However, I think a lot of customers don't know this and will run into this issue. I mean, if you think about it, just one time you don't get credits sent to you by the receiver(s), and block in FC.down() and you will block forever !
> Solution: introduce a 'minimal' OOB flag into messages, and when an OOB message is received, use a separate thread to pass it up the stack. Since we don't have many OOB messages, there shouldn't be too many threads spawned.
> We may also have to add special handling for OOB message in UNICAST.up() and NAKACK.up() as we acquire per-sender-locks there.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.jboss.com/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira