[JBoss JIRA] (WFLY-12822) Undertow Deadlock
by Brian Stansberry (Jira)
[ https://issues.redhat.com/browse/WFLY-12822?page=com.atlassian.jira.plugi... ]
Brian Stansberry commented on WFLY-12822:
-----------------------------------------
This happened again:
https://ci.wildfly.org/viewLog.html?buildId=182542&buildTypeId=WF_PullReq...
That was one of the tests of https://github.com/wildfly/wildfly/pull/12930
> Undertow Deadlock
> -----------------
>
> Key: WFLY-12822
> URL: https://issues.redhat.com/browse/WFLY-12822
> Project: WildFly
> Issue Type: Bug
> Components: Web (Undertow)
> Reporter: Brian Stansberry
> Assignee: Flavia Rainone
> Priority: Critical
> Labels: downstream_dependency
>
> Testsuite hang:
> https://ci.wildfly.org/viewLog.html?buildId=175908&buildTypeId=WF_PullReq...
> {code}
> 2019-11-22 19:51:16
> Full thread dump Java HotSpot(TM) Server VM (25.152-b16 mixed mode):
> "Attach Listener" #223 daemon prio=9 os_prio=0 tid=0xc0af2400 nid=0x40bf waiting on condition [0x00000000]
> java.lang.Thread.State: RUNNABLE
> "ServerService Thread Pool -- 78" #191 prio=5 os_prio=0 tid=0xb8678800 nid=0xb78 waiting on condition [0xbacba000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc714cdf8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> "pool-13-thread-1" #162 prio=5 os_prio=0 tid=0xbdad1800 nid=0x8a1 waiting on condition [0xbad0b000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xe2479960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> "ServerService Thread Pool -- 77" #153 prio=5 os_prio=0 tid=0xc4de7400 nid=0x896 waiting on condition [0xbac69000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc714cdf8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> "default task-2" #142 prio=5 os_prio=0 tid=0xc6848800 nid=0x872 waiting on condition [0xbac18000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc89b31d8> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "default task-1" #141 prio=5 os_prio=0 tid=0xbc8f1400 nid=0x871 waiting for monitor entry [0xbaa25000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at io.undertow.server.protocol.framed.AbstractFramedChannel.markWritesBroken(AbstractFramedChannel.java:883)
> - waiting to lock <0xe26b4980> (a io.undertow.protocols.http2.Http2Channel)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.channelForciblyClosed(AbstractFramedStreamSinkChannel.java:562)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.close(AbstractFramedStreamSinkChannel.java:539)
> at org.xnio.IoUtils.safeClose(IoUtils.java:152)
> at io.undertow.server.protocol.framed.AbstractFramedChannel.queueFrame(AbstractFramedChannel.java:728)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.queueFinalFrame(AbstractFramedStreamSinkChannel.java:257)
> - locked <0xe26ef4b8> (a java.lang.Object)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.shutdownWrites(AbstractFramedStreamSinkChannel.java:240)
> - locked <0xe26ef4b8> (a java.lang.Object)
> at io.undertow.protocols.http2.Http2Channel.sendGoAway(Http2Channel.java:822)
> at io.undertow.protocols.http2.Http2Channel.sendGoAway(Http2Channel.java:809)
> at io.undertow.protocols.http2.Http2Channel.removeStreamSink(Http2Channel.java:968)
> at io.undertow.protocols.http2.Http2StreamSinkChannel.channelForciblyClosed(Http2StreamSinkChannel.java:68)
> at io.undertow.protocols.http2.Http2DataStreamSinkChannel.channelForciblyClosed(Http2DataStreamSinkChannel.java:322)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.close(AbstractFramedStreamSinkChannel.java:539)
> at org.xnio.IoUtils.safeClose(IoUtils.java:152)
> at io.undertow.server.protocol.framed.AbstractFramedChannel.queueFrame(AbstractFramedChannel.java:728)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.queueFinalFrame(AbstractFramedStreamSinkChannel.java:257)
> - locked <0xe26b71b0> (a java.lang.Object)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.shutdownWrites(AbstractFramedStreamSinkChannel.java:240)
> - locked <0xe26b71b0> (a java.lang.Object)
> at org.xnio.conduits.StreamSinkChannelWrappingConduit.terminateWrites(StreamSinkChannelWrappingConduit.java:75)
> at org.xnio.conduits.ConduitStreamSinkChannel.shutdownWrites(ConduitStreamSinkChannel.java:178)
> at io.undertow.channels.DetachableStreamSinkChannel.shutdownWrites(DetachableStreamSinkChannel.java:79)
> at io.undertow.io.UndertowOutputStream.close(UndertowOutputStream.java:347)
> at io.undertow.io.UndertowOutputStream.updateWritten(UndertowOutputStream.java:266)
> at io.undertow.io.UndertowOutputStream.write(UndertowOutputStream.java:209)
> at io.undertow.io.UndertowOutputStream.write(UndertowOutputStream.java:105)
> at io.undertow.io.BlockingSenderImpl.send(BlockingSenderImpl.java:131)
> at io.undertow.io.BlockingSenderImpl.send(BlockingSenderImpl.java:166)
> at io.undertow.server.handlers.error.SimpleErrorPageHandler$1.handleDefaultResponse(SimpleErrorPageHandler.java:68)
> at io.undertow.server.HttpServerExchange.endExchange(HttpServerExchange.java:1608)
> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:401)
> at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
> at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
> at java.lang.Thread.run(Thread.java:748)
> "expiration-thread--p3-t1" #131 daemon prio=1 os_prio=0 tid=0xbd8b2400 nid=0x863 waiting on condition [0xbaead000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xe21794d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49)
> at org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70)
> at java.lang.Thread.run(Thread.java:748)
> "Reference Reaper #3" #129 daemon prio=5 os_prio=0 tid=0xc1bee400 nid=0x85e in Object.wait() [0xbbead000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
> - locked <0xe24ac228> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
> at org.wildfly.common.ref.References$ReaperThread.run(References.java:76)
> "Reference Reaper #2" #128 daemon prio=5 os_prio=0 tid=0xc012e400 nid=0x85d in Object.wait() [0xbbefe000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
> - locked <0xe24ac228> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
> at org.wildfly.common.ref.References$ReaperThread.run(References.java:76)
> "Reference Reaper #1" #127 daemon prio=5 os_prio=0 tid=0xc012d000 nid=0x85c in Object.wait() [0xbc05b000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
> - locked <0xe24ac228> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
> at org.wildfly.common.ref.References$ReaperThread.run(References.java:76)
> "management task-2" #125 prio=5 os_prio=0 tid=0xbdcde400 nid=0x83e waiting on condition [0xbc0fd000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8a77ca0> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "management task-1" #124 prio=5 os_prio=0 tid=0xbbf30800 nid=0x83d waiting on condition [0xbc35c000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8a77ca0> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "pool-8-thread-1" #123 prio=5 os_prio=0 tid=0xbbf2e800 nid=0x83c waiting on condition [0xbc3ad000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc898fc98> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> "DeploymentScanner-threads - 2" #121 prio=5 os_prio=0 tid=0xc3f5a000 nid=0x83a waiting on condition [0xbcaba000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8a64f80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> "Transaction Reaper Worker 0" #120 daemon prio=5 os_prio=0 tid=0xbff57400 nid=0x839 in Object.wait() [0xbcb0b000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at com.arjuna.ats.arjuna.coordinator.TransactionReaper.waitForCancellations(TransactionReaper.java:328)
> - locked <0xe2258890> (a java.util.LinkedList)
> at com.arjuna.ats.internal.arjuna.coordinator.ReaperWorkerThread.run(ReaperWorkerThread.java:65)
> "Transaction Reaper" #119 daemon prio=5 os_prio=0 tid=0xbd84a800 nid=0x837 in Object.wait() [0xbcb5c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at com.arjuna.ats.internal.arjuna.coordinator.ReaperThread.run(ReaperThread.java:90)
> - locked <0xe22586c0> (a com.arjuna.ats.arjuna.coordinator.TransactionReaper)
> "Periodic Recovery" #115 prio=5 os_prio=0 tid=0xbdfe1c00 nid=0x836 in Object.wait() [0xbcbad000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doPeriodicWait(PeriodicRecovery.java:683)
> at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:403)
> - locked <0xe227f4b8> (a java.lang.Object)
> "Transaction Expired Entry Monitor" #118 daemon prio=5 os_prio=0 tid=0xbdfe0800 nid=0x835 in Object.wait() [0xbcbfe000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xe226f588> (a com.arjuna.ats.internal.arjuna.recovery.ExpiredEntryMonitor)
> at com.arjuna.ats.internal.arjuna.recovery.ExpiredEntryMonitor.run(ExpiredEntryMonitor.java:190)
> - locked <0xe226f588> (a com.arjuna.ats.internal.arjuna.recovery.ExpiredEntryMonitor)
> "XNIO-1 Accept" #117 daemon prio=5 os_prio=0 tid=0xc4261400 nid=0x833 runnable [0xbce6c000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xe21df9e0> (a sun.nio.ch.Util$3)
> - locked <0xe21df9d0> (a java.util.Collections$UnmodifiableSet)
> - locked <0xe21df8d8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)
> "XNIO-1 I/O-1" #116 daemon prio=5 os_prio=0 tid=0xc4260400 nid=0x832 runnable [0xbcffc000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xe21df3a0> (a sun.nio.ch.Util$3)
> - locked <0xe21df390> (a java.util.Collections$UnmodifiableSet)
> - locked <0xe21df298> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)
> "Timer-1" #114 prio=5 os_prio=0 tid=0xbe20c400 nid=0x82f in Object.wait() [0xbd67a000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xc8d8bf48> (a java.util.TaskQueue)
> at java.lang.Object.wait(Object.java:502)
> at java.util.TimerThread.mainLoop(Timer.java:526)
> - locked <0xc8d8bf48> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:505)
> "default Accept" #113 prio=5 os_prio=0 tid=0xbe023400 nid=0x82e runnable [0xbdb5c000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8dabc58> (a sun.nio.ch.Util$3)
> - locked <0xc8dabc48> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8dabb50> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)
> "default I/O-8" #112 prio=5 os_prio=0 tid=0xbe022000 nid=0x82d runnable [0xbdbad000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8dae2b0> (a sun.nio.ch.Util$3)
> - locked <0xc8dae2a0> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8dae1a8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)
> "default I/O-7" #111 prio=5 os_prio=0 tid=0xbe020800 nid=0x82c runnable [0xbdbfe000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8dae8f8> (a sun.nio.ch.Util$3)
> - locked <0xc8dae8e8> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8dae7f0> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:551)
> "default I/O-6" #110 prio=5 os_prio=0 tid=0xbe01f000 nid=0x82b runnable [0xbdd69000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8db6f50> (a sun.nio.ch.Util$3)
> - locked <0xc8db6f40> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8db6e48> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:551)
> "default I/O-5" #109 prio=5 os_prio=0 tid=0xbe01c400 nid=0x82a runnable [0xbddba000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8db95a8> (a sun.nio.ch.Util$3)
> - locked <0xc8db9598> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8db94a0> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)
> "default I/O-4" #108 prio=5 os_prio=0 tid=0xbe01b000 nid=0x829 runnable [0xbde0b000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8db9bf0> (a sun.nio.ch.Util$3)
> - locked <0xc8db9be0> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8db9ae8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)
> "default I/O-3" #107 prio=5 os_prio=0 tid=0xbea2b800 nid=0x828 runnable [0xbde5c000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8dba238> (a sun.nio.ch.Util$3)
> - locked <0xc8dba228> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8dba130> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)
> "default I/O-2" #106 prio=5 os_prio=0 tid=0xbe019800 nid=0x827 runnable [0xbdead000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8dba880> (a sun.nio.ch.Util$3)
> - locked <0xc8dba870> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8dba778> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)
> "default I/O-1" #105 prio=5 os_prio=0 tid=0xbea25400 nid=0x826 waiting for monitor entry [0xbdefe000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.close(AbstractFramedStreamSinkChannel.java:525)
> - waiting to lock <0xe26b71b0> (a java.lang.Object)
> at org.xnio.IoUtils.safeClose(IoUtils.java:152)
> at io.undertow.protocols.http2.Http2Channel.closeSubChannels(Http2Channel.java:645)
> at io.undertow.server.protocol.framed.AbstractFramedChannel.close(AbstractFramedChannel.java:824)
> at org.xnio.IoUtils.safeClose(IoUtils.java:152)
> at io.undertow.protocols.http2.Http2Channel.lastDataRead(Http2Channel.java:593)
> at io.undertow.server.protocol.framed.AbstractFramedChannel.receive(AbstractFramedChannel.java:391)
> - locked <0xe26b4980> (a io.undertow.protocols.http2.Http2Channel)
> at io.undertow.server.protocol.http2.Http2ReceiveListener.handleEvent(Http2ReceiveListener.java:118)
> at io.undertow.server.protocol.http2.Http2ReceiveListener.handleEvent(Http2ReceiveListener.java:67)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener.handleEvent(AbstractFramedChannel.java:950)
> at io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener.handleEvent(AbstractFramedChannel.java:931)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
> at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
> "IdleRemover" #104 daemon prio=5 os_prio=0 tid=0xc0286c00 nid=0x825 waiting on condition [0xbe39d000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dc9790> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
> at org.jboss.jca.core.connectionmanager.pool.idle.IdleRemover$IdleRemoverRunner.run(IdleRemover.java:261)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> "ConnectionValidator" #103 daemon prio=5 os_prio=0 tid=0xbf9f8800 nid=0x824 waiting on condition [0xbe3ee000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dc9d78> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
> at org.jboss.jca.core.connectionmanager.pool.validator.ConnectionValidator$ConnectionValidatorRunner.run(ConnectionValidator.java:263)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> "Timer-0" #102 prio=5 os_prio=0 tid=0xc01a0400 nid=0x823 in Object.wait() [0xbe53f000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xc8a73050> (a java.util.TaskQueue)
> at java.lang.Object.wait(Object.java:502)
> at java.util.TimerThread.mainLoop(Timer.java:526)
> - locked <0xc8a73050> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:505)
> "management Accept" #92 prio=5 os_prio=0 tid=0xc2b20800 nid=0x820 runnable [0xbe632000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8a7f0b0> (a sun.nio.ch.Util$3)
> - locked <0xc8a7f0a0> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8a7efa8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)
> "management I/O-2" #90 prio=5 os_prio=0 tid=0xc2b1f000 nid=0x81f runnable [0xbe683000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8a7c400> (a sun.nio.ch.Util$3)
> - locked <0xc8a7c3f0> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8a7c2f8> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:551)
> "management I/O-1" #80 prio=5 os_prio=0 tid=0xc2b1bc00 nid=0x81e runnable [0xbe6d4000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> - locked <0xc8a7ea60> (a sun.nio.ch.Util$3)
> - locked <0xc8a7ea50> (a java.util.Collections$UnmodifiableSet)
> - locked <0xc8a7e958> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:551)
> "ServerService Thread Pool -- 40" #63 prio=5 os_prio=0 tid=0xc0395000 nid=0x7fb waiting on condition [0xbf4c7000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc714cdf8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> "DeploymentScanner-threads - 1" #61 prio=5 os_prio=0 tid=0xbfb52400 nid=0x7f5 waiting on condition [0xbf569000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8a64f80> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> "ServerService Thread Pool -- 9" #31 prio=5 os_prio=0 tid=0x0878e000 nid=0x7d3 waiting on condition [0xc18fd000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8a3a0d8> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> "ServerService Thread Pool -- 1" #23 prio=5 os_prio=0 tid=0xc2e37400 nid=0x7ca waiting on condition [0xc2aad000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc714cdf8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> at org.jboss.threads.JBossThread.run(JBossThread.java:485)
> "DestroyJavaVM" #22 prio=5 os_prio=0 tid=0xf6508800 nid=0x7af waiting on condition [0x00000000]
> java.lang.Thread.State: RUNNABLE
> "MSC service thread 1-8" #20 prio=5 os_prio=0 tid=0xc2103800 nid=0x7c8 waiting on condition [0xc2afe000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dca228> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "MSC service thread 1-7" #19 prio=5 os_prio=0 tid=0xc2102000 nid=0x7c7 waiting on condition [0xc2c5c000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dca228> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "MSC service thread 1-6" #18 prio=5 os_prio=0 tid=0xc2101000 nid=0x7c6 waiting on condition [0xc2cad000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dca228> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "MSC service thread 1-5" #17 prio=5 os_prio=0 tid=0xc4d3ec00 nid=0x7c5 waiting on condition [0xc29af000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dca228> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "MSC service thread 1-4" #16 prio=5 os_prio=0 tid=0xc4d3d400 nid=0x7c4 waiting on condition [0xc2a00000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dca228> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "MSC service thread 1-3" #15 prio=5 os_prio=0 tid=0xc4d2e000 nid=0x7c3 waiting on condition [0xc2a51000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dca228> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "MSC service thread 1-2" #14 prio=5 os_prio=0 tid=0xc2d48800 nid=0x7c2 waiting on condition [0xc2cfe000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dca228> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "MSC service thread 1-1" #13 prio=5 os_prio=0 tid=0xc2d46800 nid=0x7c1 waiting on condition [0xc3d7f000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc8dca228> (a org.jboss.threads.EnhancedQueueExecutor)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1409)
> at java.lang.Thread.run(Thread.java:748)
> "Reference Reaper" #9 daemon prio=5 os_prio=0 tid=0xc465b800 nid=0x7bd in Object.wait() [0xc45fe000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xc7d105d8> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
> - locked <0xc7d105d8> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
> at org.jboss.modules.ref.References$ReaperThread.run(References.java:64)
> "Service Thread" #8 daemon prio=9 os_prio=0 tid=0xc4eab400 nid=0x7bb runnable [0x00000000]
> java.lang.Thread.State: RUNNABLE
> "C1 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0xc4ea8000 nid=0x7ba waiting on condition [0x00000000]
> java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0xc4ea6800 nid=0x7b9 waiting on condition [0x00000000]
> java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0xc4ea4c00 nid=0x7b8 waiting on condition [0x00000000]
> java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0xc4ea3000 nid=0x7b7 runnable [0x00000000]
> java.lang.Thread.State: RUNNABLE
> "Finalizer" #3 daemon prio=8 os_prio=0 tid=0xc4e89000 nid=0x7b6 in Object.wait() [0xc697d000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
> - locked <0xc7360118> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
> "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0xc4e86000 nid=0x7b5 in Object.wait() [0xc6c7d000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
> - locked <0xc704c810> (a java.lang.ref.Reference$Lock)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
> "VM Thread" os_prio=0 tid=0xc4e81800 nid=0x7b4 runnable
> "GC task thread#0 (ParallelGC)" os_prio=0 tid=0xf6512000 nid=0x7b0 runnable
> "GC task thread#1 (ParallelGC)" os_prio=0 tid=0xf6513400 nid=0x7b1 runnable
> "GC task thread#2 (ParallelGC)" os_prio=0 tid=0xf6514800 nid=0x7b2 runnable
> "GC task thread#3 (ParallelGC)" os_prio=0 tid=0xf6515c00 nid=0x7b3 runnable
> "VM Periodic Task Thread" os_prio=0 tid=0xc4eaf400 nid=0x7bc waiting on condition
> JNI global references: 1879
> Found one Java-level deadlock:
> =============================
> "default task-1":
> waiting to lock monitor 0xc3cba830 (object 0xe26b4980, a io.undertow.protocols.http2.Http2Channel),
> which is held by "default I/O-1"
> "default I/O-1":
> waiting to lock monitor 0xc3cba89c (object 0xe26b71b0, a java.lang.Object),
> which is held by "default task-1"
> Java stack information for the threads listed above:
> ===================================================
> "default task-1":
> at io.undertow.server.protocol.framed.AbstractFramedChannel.markWritesBroken(AbstractFramedChannel.java:883)
> - waiting to lock <0xe26b4980> (a io.undertow.protocols.http2.Http2Channel)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.channelForciblyClosed(AbstractFramedStreamSinkChannel.java:562)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.close(AbstractFramedStreamSinkChannel.java:539)
> at org.xnio.IoUtils.safeClose(IoUtils.java:152)
> at io.undertow.server.protocol.framed.AbstractFramedChannel.queueFrame(AbstractFramedChannel.java:728)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.queueFinalFrame(AbstractFramedStreamSinkChannel.java:257)
> - locked <0xe26ef4b8> (a java.lang.Object)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.shutdownWrites(AbstractFramedStreamSinkChannel.java:240)
> - locked <0xe26ef4b8> (a java.lang.Object)
> at io.undertow.protocols.http2.Http2Channel.sendGoAway(Http2Channel.java:822)
> at io.undertow.protocols.http2.Http2Channel.sendGoAway(Http2Channel.java:809)
> at io.undertow.protocols.http2.Http2Channel.removeStreamSink(Http2Channel.java:968)
> at io.undertow.protocols.http2.Http2StreamSinkChannel.channelForciblyClosed(Http2StreamSinkChannel.java:68)
> at io.undertow.protocols.http2.Http2DataStreamSinkChannel.channelForciblyClosed(Http2DataStreamSinkChannel.java:322)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.close(AbstractFramedStreamSinkChannel.java:539)
> at org.xnio.IoUtils.safeClose(IoUtils.java:152)
> at io.undertow.server.protocol.framed.AbstractFramedChannel.queueFrame(AbstractFramedChannel.java:728)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.queueFinalFrame(AbstractFramedStreamSinkChannel.java:257)
> - locked <0xe26b71b0> (a java.lang.Object)
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.shutdownWrites(AbstractFramedStreamSinkChannel.java:240)
> - locked <0xe26b71b0> (a java.lang.Object)
> at org.xnio.conduits.StreamSinkChannelWrappingConduit.terminateWrites(StreamSinkChannelWrappingConduit.java:75)
> at org.xnio.conduits.ConduitStreamSinkChannel.shutdownWrites(ConduitStreamSinkChannel.java:178)
> at io.undertow.channels.DetachableStreamSinkChannel.shutdownWrites(DetachableStreamSinkChannel.java:79)
> at io.undertow.io.UndertowOutputStream.close(UndertowOutputStream.java:347)
> at io.undertow.io.UndertowOutputStream.updateWritten(UndertowOutputStream.java:266)
> at io.undertow.io.UndertowOutputStream.write(UndertowOutputStream.java:209)
> at io.undertow.io.UndertowOutputStream.write(UndertowOutputStream.java:105)
> at io.undertow.io.BlockingSenderImpl.send(BlockingSenderImpl.java:131)
> at io.undertow.io.BlockingSenderImpl.send(BlockingSenderImpl.java:166)
> at io.undertow.server.handlers.error.SimpleErrorPageHandler$1.handleDefaultResponse(SimpleErrorPageHandler.java:68)
> at io.undertow.server.HttpServerExchange.endExchange(HttpServerExchange.java:1608)
> at io.undertow.server.Connectors.executeRootHandler(Connectors.java:401)
> at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
> at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
> at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
> at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
> at java.lang.Thread.run(Thread.java:748)
> "default I/O-1":
> at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.close(AbstractFramedStreamSinkChannel.java:525)
> - waiting to lock <0xe26b71b0> (a java.lang.Object)
> at org.xnio.IoUtils.safeClose(IoUtils.java:152)
> at io.undertow.protocols.http2.Http2Channel.closeSubChannels(Http2Channel.java:645)
> at io.undertow.server.protocol.framed.AbstractFramedChannel.close(AbstractFramedChannel.java:824)
> at org.xnio.IoUtils.safeClose(IoUtils.java:152)
> at io.undertow.protocols.http2.Http2Channel.lastDataRead(Http2Channel.java:593)
> at io.undertow.server.protocol.framed.AbstractFramedChannel.receive(AbstractFramedChannel.java:391)
> - locked <0xe26b4980> (a io.undertow.protocols.http2.Http2Channel)
> at io.undertow.server.protocol.http2.Http2ReceiveListener.handleEvent(Http2ReceiveListener.java:118)
> at io.undertow.server.protocol.http2.Http2ReceiveListener.handleEvent(Http2ReceiveListener.java:67)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener.handleEvent(AbstractFramedChannel.java:950)
> at io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener.handleEvent(AbstractFramedChannel.java:931)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
> at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
> Found 1 deadlock.
> {code}
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 3 months
[JBoss JIRA] (DROOLS-4952) drools-7.31 expert missing/ dropped ProjectClassLoader class?
by John Harrison (Jira)
John Harrison created DROOLS-4952:
-------------------------------------
Summary: drools-7.31 expert missing/ dropped ProjectClassLoader class?
Key: DROOLS-4952
URL: https://issues.redhat.com/browse/DROOLS-4952
Project: Drools
Issue Type: Bug
Components: core engine
Affects Versions: 7.31.0.Final
Reporter: John Harrison
Assignee: Mario Fusco
I'm compiling/ running rules using expert (drools-7.31) and java 13.0.1, and getting errors like the following:
{code:java}
java.lang.NoClassDefFoundError: org/drools/reflective/classloader/ProjectClassLoader
[context - 1447]: at org.drools.core.impl.KnowledgeBaseFactory.newKnowledgeBase(KnowledgeBaseFactory.java:106)
[context - 1447]: at org.drools.core.impl.KnowledgeBaseFactory.newKnowledgeBase(KnowledgeBaseFactory.java:80)
[context - 1447]: at org.drools.core.impl.KnowledgeBaseFactory.newKnowledgeBase(KnowledgeBaseFactory.java:64)
[context - 1447]: at Reason$Descriptor.reason(Reason.java:459)
[context - 1447]: at Reason.lambda$run$1(Reason.java:82)
[context - 1447]: at io.javalin.core.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:22)
[context - 1447]: at io.javalin.http.JavalinServlet$addHandler$protectedHandler$1.handle(JavalinServlet.kt:116)
[context - 1447]: at io.javalin.http.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:45)
[context - 1447]: at io.javalin.http.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:24)
[context - 1447]: at io.javalin.http.JavalinServlet$service$1.invoke(JavalinServlet.kt:123)
[context - 1447]: at io.javalin.http.JavalinServlet$service$2.invoke(JavalinServlet.kt:40)
[context - 1447]: at io.javalin.http.JavalinServlet.service(JavalinServlet.kt:75)
[context - 1447]: at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[context - 1447]: at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867)
[context - 1447]: at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542)
[context - 1447]: at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
[context - 1447]: at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
[context - 1447]: at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
[context - 1447]: at io.javalin.core.JavalinServer$start$httpHandler$1.doHandle(JavalinServer.kt:53)
[context - 1447]: at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
[context - 1447]: at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
[context - 1447]: at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
[context - 1447]: at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
[context - 1447]: at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
[context - 1447]: at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
[context - 1447]: at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:61)
[context - 1447]: at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174)
[context - 1447]: at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[context - 1447]: at org.eclipse.jetty.server.Server.handle(Server.java:502)
[context - 1447]: at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
[context - 1447]: at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
[context - 1447]: at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
[context - 1447]: at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
[context - 1447]: at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
[context - 1447]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
[context - 1447]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
[context - 1447]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
[context - 1447]: at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
[context - 1447]: at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
[context - 1447]: at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
[context - 1447]: at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
[context - 1447]: at java.base/java.lang.Thread.run(Thread.java:830)
[context - 1447]: Caused by: java.lang.ClassNotFoundException: org.drools.reflective.classloader.ProjectClassLoader
[context - 1447]: at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602)
[context - 1447]: at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
[context - 1447]: at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
[context - 1447]: ... 42 more
{code}
Is this class missing or removed from the distribution? The related release notes don't mention anything, and things work fine in drools-7.30 (class located in drools-core-7.30.0.Final.jar).
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 3 months
[JBoss JIRA] (WFLY-12614) Duplicated ConstraintViolation message
by Robin Schimpf (Jira)
[ https://issues.redhat.com/browse/WFLY-12614?page=com.atlassian.jira.plugi... ]
Robin Schimpf commented on WFLY-12614:
--------------------------------------
I have seen that the 19.0.0.Beta1 contains RestEasy 3.10.0.Final by WFLY-12712 which is the fix version for the linked fix of this issue. I retested the attached example with the beta version and it works as expected.
I think this issue can be marked as resolved now.
> Duplicated ConstraintViolation message
> --------------------------------------
>
> Key: WFLY-12614
> URL: https://issues.redhat.com/browse/WFLY-12614
> Project: WildFly
> Issue Type: Bug
> Components: Bean Validation, REST
> Affects Versions: 17.0.1.Final, 18.0.0.Beta1
> Reporter: Robin Schimpf
> Assignee: Ronald Sigal
> Priority: Major
> Attachments: wildfly-bug.zip
>
>
> We currently are upgrading our application from Wildfly 13 to Wildfly 17.0.1 and are receiving duplicated constraint violations in the response of an invalid request.
> Interestingly Wildfly 13 also seems to be not behaving correctly on the third request but in another way. There is the violation exception returned in the response instead of the duplicated value Wildfly 17 is returning now.
> I also tested this on the currently available Wildfly 18 Beta 1 and the issue is also reproducible.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 3 months
[JBoss JIRA] (WFLY-12950) [EAT] Multiversion Asynchronous EJB test
by Panagiotis Sotiropoulos (Jira)
[ https://issues.redhat.com/browse/WFLY-12950?page=com.atlassian.jira.plugi... ]
Panagiotis Sotiropoulos edited comment on WFLY-12950 at 1/18/20 2:59 AM:
-------------------------------------------------------------------------
Note : Reduced async calls in order to eliminate the concurrency faults, as discussed (not related).
was (Author: takis):
Note : Reduced async calls in order to eliminate the concurrency faults, as discussed.
> [EAT] Multiversion Asynchronous EJB test
> ----------------------------------------
>
> Key: WFLY-12950
> URL: https://issues.redhat.com/browse/WFLY-12950
> Project: WildFly
> Issue Type: Task
> Components: Test Suite
> Reporter: Panagiotis Sotiropoulos
> Assignee: Panagiotis Sotiropoulos
> Priority: Major
>
> Multiversion Test for issue "Calling Asynchronous EJB will use the propagated caller transaction which is not according to the specification"
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 3 months
[JBoss JIRA] (WFLY-12950) [EAT] Multiversion Asynchronous EJB test
by Panagiotis Sotiropoulos (Jira)
[ https://issues.redhat.com/browse/WFLY-12950?page=com.atlassian.jira.plugi... ]
Panagiotis Sotiropoulos edited comment on WFLY-12950 at 1/18/20 2:59 AM:
-------------------------------------------------------------------------
Note : Reduced async calls in order to eliminate the concurrency faults, as discussed (not related to this issue).
was (Author: takis):
Note : Reduced async calls in order to eliminate the concurrency faults, as discussed (not related).
> [EAT] Multiversion Asynchronous EJB test
> ----------------------------------------
>
> Key: WFLY-12950
> URL: https://issues.redhat.com/browse/WFLY-12950
> Project: WildFly
> Issue Type: Task
> Components: Test Suite
> Reporter: Panagiotis Sotiropoulos
> Assignee: Panagiotis Sotiropoulos
> Priority: Major
>
> Multiversion Test for issue "Calling Asynchronous EJB will use the propagated caller transaction which is not according to the specification"
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 3 months
[JBoss JIRA] (WFLY-12950) [EAT] Multiversion Asynchronous EJB test
by Panagiotis Sotiropoulos (Jira)
[ https://issues.redhat.com/browse/WFLY-12950?page=com.atlassian.jira.plugi... ]
Panagiotis Sotiropoulos edited comment on WFLY-12950 at 1/18/20 2:48 AM:
-------------------------------------------------------------------------
I have forgotten the relation (WFLY-12958) ...
was (Author: takis):
I have forgot the relation (WFLY-12958) ...
> [EAT] Multiversion Asynchronous EJB test
> ----------------------------------------
>
> Key: WFLY-12950
> URL: https://issues.redhat.com/browse/WFLY-12950
> Project: WildFly
> Issue Type: Task
> Components: Test Suite
> Reporter: Panagiotis Sotiropoulos
> Assignee: Panagiotis Sotiropoulos
> Priority: Major
>
> Multiversion Test for issue "Calling Asynchronous EJB will use the propagated caller transaction which is not according to the specification"
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 3 months
[JBoss JIRA] (DROOLS-4930) [DMN Designer] Remove UUID from Alerts panel validation messages
by Roger Martinez (Jira)
[ https://issues.redhat.com/browse/DROOLS-4930?page=com.atlassian.jira.plug... ]
Roger Martinez commented on DROOLS-4930:
----------------------------------------
Hey [~jomarko] [~manstis]
I'd been testing BPMN and it seems the visual feedback for validation is "partially" broken once saving, so probably it's why [~jomarko] also says it's not working for DMN. Here is the issue I just reported:
https://issues.redhat.com/browse/JBPM-8981
I don't really know the details about the use of the UUIDS for DMN, but this feature is generic and you could take benefit from, in case you think it makes sense.
I just works as:
# Validation process generates validation messages
# Validation messages contain the node/s UUID's (although we can not expose those on the UI or alerts panel, they should be there in the validation message model)
# After validating (or saving) -> Stunner iterates over the UUID's for all the validation messages, obtain the nodes/connectors by the respective UUID, and perform the highlighting on the canvas side (red borders)
So rather than the [JBPM-8981|https://issues.redhat.com/browse/JBPM-8981], which has to be fixed, if your messages & nodes UUID's correspond, you should be able to see errors as highlighted nodes.
PS: Anyway, the whole usecase is not yet complete, as you cannot click on some message in the Alerts Panel, nothing happens, no node is selected, so once having several error messages and nodes highlighted is also not easy to identify each one.
> [DMN Designer] Remove UUID from Alerts panel validation messages
> -----------------------------------------------------------------
>
> Key: DROOLS-4930
> URL: https://issues.redhat.com/browse/DROOLS-4930
> Project: Drools
> Issue Type: Enhancement
> Components: DMN Editor
> Affects Versions: 7.31.0.Final
> Reporter: Michael Anstis
> Assignee: Michael Anstis
> Priority: Major
> Labels: drools-tools
> Attachments: Screenshot from 2020-01-17 07-41-57.png
>
>
> Validation of a DMN diagram can lead to messages being shown in the Alerts Panel.
> Stunner is appending the UUID to the message. See [CoreTranslationMessages.getElementValidationMessage()|https://github.com/...]
> [~tirelli] has requested their removal.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 3 months
[JBoss JIRA] (WFLY-11943) Statistics of REST endpoints
by James Perkins (Jira)
[ https://issues.redhat.com/browse/WFLY-11943?page=com.atlassian.jira.plugi... ]
James Perkins updated WFLY-11943:
---------------------------------
Fix Version/s: 19.0.0.Beta2
(was: 19.0.0.Beta1)
> Statistics of REST endpoints
> ----------------------------
>
> Key: WFLY-11943
> URL: https://issues.redhat.com/browse/WFLY-11943
> Project: WildFly
> Issue Type: Feature Request
> Components: REST
> Affects Versions: 17.0.0.Final
> Reporter: r searls
> Assignee: r searls
> Priority: Major
> Labels: affects-model
> Fix For: 19.0.0.Beta2
>
>
> Provide statistics on REST endpoints in the jaxrs subsystem.
> CLI support "read-resource" operation for jax-rs subsystem in deployment:
> /deployment=jaxrs-eap.war/subsystem=jaxrs:read-resource
> Statistics should be added to results of this operation in this RFE.
> There needs to be option to enable or disable collecting of these statistics, because collecting of statistics does probably some performance drop.
> The change of statistic settings should not require server reload or restart.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years, 3 months