[Artemis 2.x Upgrade] File leak - Artemis crashes on Critical IO Error, shutting down the server. file=NIOSequentialFile .../largemessages/55851.msg, message=.../largemessages/55851.msg (Too many open files) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Key: WFLY-10022 URL: https://issues.jboss.org/browse/WFLY-10022 Project: WildFly Issue Type: Bug Components: JMS Reporter: Miroslav Novak Assignee: Martyn Taylor Priority: Blocker Labels: activemq, feature-branch-blocker There is file leak in Artemis master branch (577b62d5210cdcc0f86ab9bb1b24e944c877dfe7) when large messages are consumed from topic. This is regression against Artemis 1.5.x. Test Scenario: * Start WF12 (Jeff's WF WFLY-9407_upgrade_artemis_2.5.0 branch - 06c878a313d3cad323889d017e60fd5533204d1a) with deployed "testTopic" * create durable subscriptions on testTopic * start sending large messages on testTopic * subscribers start to consume one by one so there is a huge difference in number of consumed messages between subscriptions Pass Criteria: Subscribers must receive correct number of messages. Actual Result: Artemis server shutdown/crashes itself on (ulimit for max number of open files was set to 8196): {code} 15:21:54,717 WARN [org.apache.activemq.artemis.core.server] (Thread-21 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@1787a3ac)) AMQ222010: Critical IO Error, shutting down the server. file=NIOSequentialFile /home/mnovak/hornetq_eap6_dev/internal/eap-tests-hornetq/scripts/journal-A/largemessages/55851.msg, message=/home/mnovak/hornetq_eap6_dev/internal/eap-tests-hornetq/scripts/journal-A/largemessages/55851.msg (Too many open files): ActiveMQIOErrorException[errorType=IO_ERROR message=/home/mnovak/hornetq_eap6_dev/internal/eap-tests-hornetq/scripts/journal-A/largemessages/55851.msg (Too many open files)] at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:87) [artemis-journal-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:73) [artemis-journal-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.openFile(LargeServerMessageImpl.java:397) [artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.validateFile(LargeServerMessageImpl.java:376) [artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.createLargeMessage(JournalStorageManager.java:495) [artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.sendLarge(ServerSessionPacketHandler.java:937) [artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.slowPacketHandler(ServerSessionPacketHandler.java:302) [artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onMessagePacket(ServerSessionPacketHandler.java:281) [artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:33) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_131] at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] Caused by: java.io.FileNotFoundException: /home/mnovak/hornetq_eap6_dev/internal/eap-tests-hornetq/scripts/journal-A/largemessages/55851.msg (Too many open files) at java.io.RandomAccessFile.open0(Native Method) [rt.jar:1.8.0_131] at java.io.RandomAccessFile.open(RandomAccessFile.java:316) [rt.jar:1.8.0_131] at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) [rt.jar:1.8.0_131] at org.apache.activemq.artemis.core.io.nio.NIOSequentialFile.open(NIOSequentialFile.java:79) [artemis-journal-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT] ... 15 more {code}