[jbosscache-issues] [JBoss JIRA] Resolved: (JBCACHE-1569) JDBCCacheLoader can cause deadlocks

JDBCCacheLoader can cause deadlocks ----------------------------------- Key: JBCACHE-1569 URL: https://jira.jboss.org/jira/browse/JBCACHE-1569 Project: JBoss Cache Issue Type: Bug Security Level: Public(Everyone can see) Components: Cache loaders Affects Versions: 3.2.1.GA Reporter: Nikolay Zamosenchuk Assignee: Manik Surtani Fix For: 3.2.3.CR1, 3.2.3.GA Attachments: jbosscache-core-JDBCCacheLoader.patch Lets have a look at JDBCCacheLoader.java:243. Method remove(Fqn fqn). It acquires CONNECTION and then tries to acquire LOCK. Lets have a look at JDBCCacheLoader.java:138. Method put(Fqn name, Map attributes). It acquires LOCK and then calls addNewSubtree(..) or updateNode(..) and they are getting CONNECTION. USECASE. Concurrent removes and puts to/from cache with JDBC cache loader, having fixed pool of database connections. One thread invokes JDBCCacheLoader.remove(Fqn fqn) and: 1. Acquires CONNECTION to DB from pool. 2. Tries to LOCK fqn, but it is locked by another thread. now Connection is taken from pool, but is not currently used, cause waiting a lock. Another thread invokes JDBCCacheLoader.put(Fqn name, Map attributes) and: 1. Acquires LOCK on fqn 2. invokes addNewSubtree(..) or updateNode(..), and one of those methods tries to get CONNECTION from pool. But pool exhausted and deadlock appears. Imagine that we have pool size = 1. So classic deadlock takes place (L-lock resource, C - connection resource): First thread: gets C, L perform operations release C,L Second thread: gets L, C. perform operations release C,L (1st acquires C and 2nd acquires L, then 1st waits for L and second waits for C) When we have pool size 50..100 and many (900) threads such situation appears very often. Must admit, that AdjListJDBCCacheLoader.getChildrenNames(Fqn fqn) (line : 190), also opens connection and then tries to acquire lock on fqn. Possible way to solve is to acquire lock, before opening the connection to DB in methods: AdjListJDBCCacheLoader.getChildrenNames(...) (line : 190) JDBCCacheLoader.remove(...) (line : 254)