hanging thread -------------- Key: JBTM-2037 URL: https://issues.jboss.org/browse/JBTM-2037 Project: JBoss Transaction Manager Issue Type: Bug Security Level: Public(Everyone can see) Environment: windows server 2008 R2 Datacenter - service pack 1 wildfly 8.0.0.Beta17 build 796 Reporter: Admin FlirtyMob Assignee: Tom Jenkinson Thread still hangin after several hours and spotted through jstack: Thread 150: (state = IN_NATIVE) - java.net.SocketInputStream.socketRead0(java.io.FileDescriptor, byte[], int, int, int) @bci=0 (Interpreted frame) - java.net.SocketInputStream.read(byte[], int, int, int) @bci=87, line=152 (Interpreted frame) - java.net.SocketInputStream.read(byte[], int, int) @bci=11, line=122 (Interpreted frame) - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=273 (Interpreted frame) - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=334 (Interpreted frame) - org.apache.commons.httpclient.ChunkedInputStream.read(byte[], int, int) @bci=71, line=182 (Interpreted frame) - java.io.FilterInputStream.read(byte[], int, int) @bci=7, line=133 (Interpreted frame) - org.apache.commons.httpclient.AutoCloseInputStream.read(byte[], int, int) @bci=14, line=108 (Interpreted frame) - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=283 (Interpreted frame) - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=325 (Interpreted frame) - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=177 (Interpreted frame) - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Interpreted frame) - java.io.BufferedReader.fill() @bci=145, line=154 (Interpreted frame) - java.io.BufferedReader.readLine(boolean) @bci=44, line=317 (Compiled frame) - java.io.BufferedReader.readLine() @bci=2, line=382 (Interpreted frame) - com.flirtymob.chat.util.httpclient.HttpClientUtil.convertInputStreamToString(java.io.InputStream) @bci=20, line=19 (Interpreted frame) - com.flirtymob.chat.util.httpclient.HttpClientUtil.getResponseBodyAsString(org.apache.commons.httpclient.HttpMethod) @bci=14, line=48 (Interpreted frame) - com.flirtymob.fxrate.FxrateService.getXEForex(java.lang.String, java.lang.String) @bci=96, line=145 (Interpreted frame) - com.flirtymob.fxrate.FxrateService.automaticTimeout() @bci=74, line=53 (Interpreted frame) - sun.reflect.NativeMethodAccessorImpl.invoke0(java.lang.reflect.Method, java.lang.Object, java.lang.Object[]) @bci=0 (Interpreted frame) - sun.reflect.NativeMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=87, line=57 (Interpreted frame) - sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=6, line=43 (Interpreted frame) - java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object[]) @bci=57, line=606 (Interpreted frame) - org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=39, line=52 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.WeavedInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=51, line=53 (Interpreted frame) - org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(org.jboss.invocation.InterceptorContext) @bci=23, line=61 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.InterceptorContext$Invocation.proceed() @bci=4, line=406 (Interpreted frame) - org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(javax.interceptor.InvocationContext, javax.enterprise.inject.spi.InterceptionType, org.jboss.as.weld.ejb.WeldInterceptorInstances, org.jboss.weld.ejb.spi.InterceptorBindings) @bci=32, line=82 (Interpreted frame) - org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=106, line=95 (Interpreted frame) - org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(org.jboss.invocation.InterceptorContext) @bci=23, line=61 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.WeavedInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=51, line=53 (Interpreted frame) - org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(org.jboss.invocation.InterceptorContext) @bci=23, line=61 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=16, line=43 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.InterceptorContext$Invocation.proceed() @bci=4, line=406 (Interpreted frame) - org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=234, line=104 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.InterceptorContext$Invocation.proceed() @bci=4, line=406 (Interpreted frame) - org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(javax.interceptor.InvocationContext) @bci=61, line=55 (Interpreted frame) - org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=5, line=84 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=8, line=45 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.InitialInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=6, line=21 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.ChainedInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=19, line=61 (Interpreted frame) - org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=58, line=53 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=40, line=52 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(org.jboss.invocation.InterceptorContext, javax.transaction.TransactionManager, org.jboss.as.ejb3.component.EJBComponent) @bci=24, line=273 (Interpreted frame) - org.jboss.as.ejb3.tx.CMTTxInterceptor.required(org.jboss.invocation.InterceptorContext, org.jboss.as.ejb3.component.EJBComponent, int) @bci=45, line=340 (Interpreted frame) - org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=110, line=239 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=5, line=41 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(org.jboss.invocation.InterceptorContext) @bci=44, line=64 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=15, line=50 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=42, line=55 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=46, line=64 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.InterceptorContext.run() @bci=1, line=325 (Interpreted frame) - org.wildfly.security.manager.WildFlySecurityManager.doChecked(java.security.PrivilegedExceptionAction) @bci=42, line=437 (Interpreted frame) - org.jboss.invocation.AccessCheckingInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=1, line=61 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.InterceptorContext.run() @bci=1, line=325 (Interpreted frame) - org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=20, line=80 (Interpreted frame) - org.jboss.invocation.InterceptorContext.proceed() @bci=26, line=309 (Interpreted frame) - org.jboss.invocation.ChainedInterceptor.processInvocation(org.jboss.invocation.InterceptorContext) @bci=19, line=61 (Interpreted frame) - org.jboss.as.ejb3.timerservice.TimedObjectInvokerImpl.callTimeout(org.jboss.as.ejb3.timerservice.TimerImpl, java.lang.reflect.Method) @bci=197, line=104 (Interpreted frame) - org.jboss.as.ejb3.timerservice.task.CalendarTimerTask.callTimeout(org.jboss.as.ejb3.timerservice.TimerImpl) @bci=45, line=61 (Interpreted frame) - org.jboss.as.ejb3.timerservice.task.TimerTask.run() @bci=426, line=168 (Interpreted frame) - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=471 (Interpreted frame) - java.util.concurrent.FutureTask.run() @bci=42, line=262 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1145 (Compiled frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=615 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=724 (Interpreted frame) - org.jboss.threads.JBossThread.run() @bci=10, line=122 (Interpreted frame) In the server logs, there is the following short after the start of the timer task : 2013-11-26 05:05:00,040 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a2714e5:33df70b6:5293c60a:63d in state RUN 2013-11-26 05:05:00,040 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a2714e5:33df70b6:5293c60a:63d invoked while multiple threads active within it. 2013-11-26 05:05:00,040 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012108: CheckedAction::check - atomic action 0:ffff0a2714e5:33df70b6:5293c60a:63d aborting with 1 threads active! 2013-11-26 05:05:00,040 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a2714e5:33df70b6:5293c60a:63d