[ https://issues.redhat.com/browse/ISPN-12570?page=com.atlassian.jira.plugi... ] Dan Berindei updated ISPN-12570: -------------------------------- Description: {{NettyTransport.stop()}} calls {{acceptedChannels.close()}} after shutting down the both the {{masterGroup}} and the {{ioGroup}}. But if a channel is still open, closing it requires submitting a task to the channel's event loop, which is now shut down. {{AbstractChannelHandlerContext.safeExecute()}} hides the rejection exception, but the {{ChannelPromise}} returned by {{channel.close()}} never completes, and the server doesn't stop. {noformat} java.util.concurrent.RejectedExecutionException: event executor terminated at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:926) at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:353) at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:346) at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:828) at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:818) at io.netty.channel.AbstractChannelHandlerContext.safeExecute(AbstractChannelHandlerContext.java:989) at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:608) at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:472) at io.netty.channel.DefaultChannelPipeline.close(DefaultChannelPipeline.java:957) at io.netty.channel.AbstractChannel.close(AbstractChannel.java:232) at io.netty.channel.group.DefaultChannelGroup.close(DefaultChannelGroup.java:342) at io.netty.channel.group.DefaultChannelGroup.close(DefaultChannelGroup.java:221) at org.infinispan.server.core.transport.NettyTransport.stop(NettyTransport.java:135) at org.infinispan.server.core.AbstractProtocolServer.stop(AbstractProtocolServer.java:202) at org.infinispan.rest.RestServer.stop(RestServer.java:100) at org.infinispan.rest.helper.RestServerHelper.stop(RestServerHelper.java:86) {noformat} {noformat} java.lang.RuntimeException: Test timed out after 300 seconds at java.base(a)11.0.9/java.lang.Object.$$BlockHound$$_wait(Native Method) at java.base(a)11.0.9/java.lang.Object.wait(Object.java) at java.base@11.0.9/java.lang.Object.wait(Object.java:328) at app//io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:274) at app//io.netty.channel.group.DefaultChannelGroupFuture.awaitUninterruptibly(DefaultChannelGroupFuture.java:178) at app//io.netty.channel.group.DefaultChannelGroupFuture.awaitUninterruptibly(DefaultChannelGroupFuture.java:41) at app//org.infinispan.server.core.transport.NettyTransport.stop(NettyTransport.java:149) at app//org.infinispan.server.core.AbstractProtocolServer.stop(AbstractProtocolServer.java:202) at app//org.infinispan.rest.RestServer.stop(RestServer.java:100) at app//org.infinispan.rest.helper.RestServerHelper.stop(RestServerHelper.java:86) {noformat} {{CacheV2ResourceTest.afterSuite()}} sometimes hangs this way. Initially I attributed the hanging to ISPN-12558, but it looks like a separate issue. was: {{NettyTransport.stop()}} calls {{acceptedChannels.close()}} after shutting down the both the {{masterGroup}} and the {{ioGroup}}. But if a channel is still open, closing it requires submitting a task to the channel's event loop, which is now shut down. {{AbstractChannelHandlerContext.safeExecute()}} hides the rejection exception, but the {{ChannelPromise}} returned by {{channel.close()}} never completes, and the server doesn't stop. {noformat} java.util.concurrent.RejectedExecutionException: event executor terminated at io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:926) at io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:353) at io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:346) at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:828) at io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:818) at io.netty.channel.AbstractChannelHandlerContext.safeExecute(AbstractChannelHandlerContext.java:989) at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:608) at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:472) at io.netty.channel.DefaultChannelPipeline.close(DefaultChannelPipeline.java:957) at io.netty.channel.AbstractChannel.close(AbstractChannel.java:232) at io.netty.channel.group.DefaultChannelGroup.close(DefaultChannelGroup.java:342) at io.netty.channel.group.DefaultChannelGroup.close(DefaultChannelGroup.java:221) at org.infinispan.server.core.transport.NettyTransport.stop(NettyTransport.java:135) at org.infinispan.server.core.AbstractProtocolServer.stop(AbstractProtocolServer.java:202) at org.infinispan.rest.RestServer.stop(RestServer.java:100) at org.infinispan.rest.helper.RestServerHelper.stop(RestServerHelper.java:86) {noformat} {noformat} at java.lang.Object.$$BlockHound$$_wait(Object.java:-2) at java.lang.Object.wait(Object.java:-1) at java.lang.Object.wait(Object.java:328) at io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:264) at io.netty.channel.group.DefaultChannelGroupFuture.awaitUninterruptibly(DefaultChannelGroupFuture.java:178) at io.netty.channel.group.DefaultChannelGroupFuture.awaitUninterruptibly(DefaultChannelGroupFuture.java:41) at org.infinispan.server.core.transport.NettyTransport.stop(NettyTransport.java:149) at org.infinispan.server.core.AbstractProtocolServer.stop(AbstractProtocolServer.java:202) at org.infinispan.rest.RestServer.stop(RestServer.java:100) at org.infinispan.rest.helper.RestServerHelper.stop(RestServerHelper.java:86) {noformat} {{CacheV2ResourceTest.afterSuite()}} sometimes hangs this way. Initially I attributed the hanging to ISPN-12558, but it looks like a separate issue. -- This message was sent by Atlassian Jira (v7.13.8#713008)