Re: [undertow-dev] InSessionMemoryManager synchronized methods

From: "Eric Peters" <epeters(a)epicor.com&gt; To: undertow-dev(a)lists.jboss.org Sent: Wednesday, 1 July, 2015 8:44:46 PM Subject: Re: [undertow-dev] InSessionMemoryManager synchronized methods I’m still trying to gauge the release process for Undertow, and where in that process this issue sits, so as to decide how to roll things out to customers on my end: - I see UNDERTOW-479 was entered for this deadlock. - I see a patch for this was pulled into Undertow 1.3.0 Beta 3 - The patch has not been back-ported to prior releases. - The Jira case is still open. What can I expect to see in the coming days? I need an answer to give management for the question of ‘How soon will the patch be deemed production-ready?’, but I’d appreciate any additional insight that can be provided. //EricP From: Eric Peters Sent: Thursday, June 25, 2015 5:23 PM To: Eric Peters; undertow-dev(a)lists.jboss.org Subject: RE: InSessionMemoryManager synchronized methods I stand corrected – This is 50% of what brought our server down. The other half of the deadlock belongs to Seam: * io.undertow.server.session.InMemorySessionManager$SessionImpl.bumpTimeout() (line: 271) * io.undertow.server.session.InMemorySessionManager$SessionImpl.getAttribute(java.lang.String) (line: 355) * io.undertow.servlet.spec.HttpSessionImpl.getAttribute(java.lang.String) * com.sun.faces.context.SessionMap.get(java.lang.Object) * org.jboss.seam.contexts.ServerConversationContext.get(java.lang.String) (line: 110) * org.jboss.seam.contexts.Contexts.lookupInStatefulContexts(java.lang.String) (line: 189) * org.jboss.seam.Component.getInstanceFromFactory(java.lang.String, org.jboss.seam.ScopeType) (line: 2088) * org.jboss.seam.Component.getInstance(java.lang.String, boolean, boolean, java.lang.Object, org.jboss.seam.ScopeType) (line: 2014) * org.jboss.seam.Component.getInstance(java.lang.String, boolean, boolean) (line: 1986) * org.jboss.seam.Component.getInstance(java.lang.String, boolean) * org.jboss.seam.Namespace.getComponentInstance(java.lang.String, boolean) org.jboss.seam.Component.getInstanceFromFactory() obtains a lock that has static scope, so nothing else can invoke getInstanceFromFacotry() while the lock is held.It then blocks on InMemorySessionManager$SessionImpl.bumpTimeout() Meanwhile, the invalidate() method blocks against the static method in getInstanceFromFactory() //EricP From: undertow-dev-bounces(a)lists.jboss.org [ mailto:undertow-dev-bounces@lists.jboss.org ] On Behalf Of Eric Peters Sent: Thursday, June 25, 2015 4:39 PM To: undertow-dev(a)lists.jboss.org Subject: [undertow-dev] InSessionMemoryManager synchronized methods Chasing after another problem I found several blocked task threads, best as I can determine: InMemorySessionmanager$SessionImpl.invalidate() is invoked. This method is synchronized, and the call blocks inside SesssionListeners.sessionDestroyed() (separate issue there…) Later, InMemorySessionmanager$SessionImpl.bumpTimeout() is invoked from another thread. This method is also synchronized, and blocks waiting for invalidate() to complete. As the first thread is indefinitely blocked inside invalidate(), this second thread also remains blocked indefinitely. In my environment it appears to make an existing problem slightly more severe. Found in 1.1.0, but it looks like it’s in both 1.1.7 and the master branch as well. Using RHEL, Java 7, Wildfly 8.2 invalidate() Thread trace (excerpt): * (blocked) - sun.misc.Unsafe.park(boolean, long) * … * org.jboss.seam.contexts.ServletLifecycle.endSession(javax.servlet.http.HttpSession) (line: 187) * org.jboss.seam.servlet.SeamListener.sessionDestroyed(javax.servlet.http.HttpSessionEvent) (line: 59) * io.undertow.servlet.core.ApplicationListeners.sessionDestroyed(javax.servlet.http.HttpSession) (line: 264) * io.undertow.servlet.core.SessionListenerBridge.sessionDestroyed(io.undertow.server.session.Session, io.undertow.server.HttpServerExchange, io.undertow.server.session.SessionListener$SessionDestroyedReason) (line: 67) * io.undertow.server.session.SessionListeners.sessionDestroyed(io.undertow.server.session.Session, io.undertow.server.HttpServerExchange, io.undertow.server.session.SessionListener$SessionDestroyedReason) (line: 61) * io.undertow.server.session.InMemorySessionManager$SessionImpl.invalidate(io.undertow.server.HttpServerExchange, io.undertow.server.session.SessionListener$SessionDestroyedReason) (line: 413) * io.undertow.server.session.InMemorySessionManager$SessionImpl.invalidate(io.undertow.server.HttpServerExchange) (line: 399) * io.undertow.servlet.spec.HttpSessionImpl.invalidate() (line: 197) * … bumpTimeout() Thread trace: * io.undertow.server.session.InMemorySessionManager$SessionImpl.bumpTimeout() (line: 271) * io.undertow.server.session.InMemorySessionManager$SessionImpl.getAttribute(java.lang.String) (line: 355) * io.undertow.servlet.util.SavedRequest.tryRestoreRequest(io.undertow.server.HttpServerExchange, javax.servlet.http.HttpSession) (line: 124) * io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(io.undertow.server.HttpServerExchange) (line: 67) * io.undertow.security.handlers.SecurityInitialHandler.handleRequest(io.undertow.server.HttpServerExchange) (line: 76) * io.undertow.server.handlers.PredicateHandler.handleRequest(io.undertow.server.HttpServerExchange) (line: 43) * org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(io.undertow.server.HttpServerExchange) (line: 61) * io.undertow.server.handlers.PredicateHandler.handleRequest(io.undertow.server.HttpServerExchange) (line: 43) * io.undertow.server.handlers.PredicateHandler.handleRequest(io.undertow.server.HttpServerExchange) (line: 43) * io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(io.undertow.server.HttpServerExchange, io.undertow.servlet.handlers.ServletChain, io.undertow.servlet.handlers.ServletRequestContext, javax.servlet.ServletRequest, javax.servlet.ServletResponse) (line: 261) * io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(io.undertow.server.HttpServerExchange, io.undertow.servlet.handlers.ServletRequestContext, io.undertow.servlet.handlers.ServletChain, javax.servlet.DispatcherType) (line: 247) * io.undertow.servlet.handlers.ServletInitialHandler.access$000(io.undertow.servlet.handlers.ServletInitialHandler, io.undertow.server.HttpServerExchange, io.undertow.servlet.handlers.ServletRequestContext, io.undertow.servlet.handlers.ServletChain, javax.servlet.DispatcherType) * io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(io.undertow.server.HttpServerExchange) (line: 166) * io.undertow.server.Connectors.executeRootHandler(io.undertow.server.HttpHandler, io.undertow.server.HttpServerExchange) (line: 197) * io.undertow.server.HttpServerExchange$1.run() (line: 759) * java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) (line: 1145) * java.util.concurrent.ThreadPoolExecutor$Worker.run() (line: 615) * java.lang.Thread.run() (line: 745) //EricP This message has been scanned for malware by Websense. www.websense.com Click here to report this email as spam. _______________________________________________ undertow-dev mailing list undertow-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/undertow-dev