[infinispan-issues] [JBoss JIRA] (ISPN-9589) Client Listener notifications can deadlock

[William Burns (Jira)]

    [ https://issues.jboss.org/browse/ISPN-9589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644200#comment-13644200 ] 

William Burns commented on ISPN-9589:
-------------------------------------

I can confirm the fix in my previous comment is the best way. Unfortunately this will add a small bit of latency per write operation as it will have to check the listeners on every request - since the information is not static like the others for CacheInfo

> Client Listener notifications can deadlock
> ------------------------------------------
>
>                 Key: ISPN-9589
>                 URL: https://issues.jboss.org/browse/ISPN-9589
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Listeners, Server
>            Reporter: William Burns
>            Priority: Critical
>
> When a node does a write and it is the primary owner it will send the client listener event. Unfortunately this can block waiting to put in the eventQueue. Unfortunately the thread that is executing this method can be the event loop for that listener, which in turn can cause it to block forever. I believe this came about after ISPN-8621.
> I have found a few ways to fix this at the moment:
> # Make the client listener async - this way it is fired on the notification thread
> # Change the listener event to check if the current thread is the event loop and subsequently fire the events directly instead of submitting it to the Executor.
> # Make the various write operations still use the worker thread pool - this was changed in the above mentioned JIRA so I am guessing we don't want to do that
> The former of the fixes seems to be much more performant, but I don't know if I like that fix. The latter still has hiccups since it reduces availability of threads to respond to socket requests.
> This is very easy to reproduce by running the ClientEventStressTest test on master as is since it only has 3 io threads (but the default is CPU * 2 - so in some cases this isn't that much)



--
This message was sent by Atlassian Jira
(v7.12.1#712002)