[jboss-jira] [JBoss JIRA] (AS7-3408) Classloading issue with HornetQLoggerFactory - problem with failback

Classloading issue with HornetQLoggerFactory - problem with failback -------------------------------------------------------------------- Key: AS7-3408 URL: https://issues.jboss.org/browse/AS7-3408 Project: Application Server 7 Issue Type: Bug Components: JMS Affects Versions: 7.1.0.CR1 Reporter: Miroslav Novak Assignee: Andy Taylor Priority: Critical Fix For: 7.1.0.Final Attachments: console-log-backup-server.txt, console-log-live-server.txt, reproducer.zip Test scenario: 1. Start two servers AS7/EAP6 servers - live and its backup in dedicated topology - each on different machine 2. Kill live server using "kill -9 ..." 3. Start live server again In step 3. there are unexpected messages in console log of live and backup server. From backup server: {code} 12:07:26,165 INFO [org.hornetq.core.server.impl.HornetQServerImpl] (Thread-78) HornetQ Server version 2.2.7.Final (HQ_2_2_7_FINAL_AS7, 121) [17700d86-45b2-11e1-a575-d48564b8e1e7] stopped 12:07:26,165 INFO [org.hornetq.core.server.impl.HornetQServerImpl] (Thread-78) unable to restart server, please kill and restart manually: java.lang.IllegalArgumentException: Could not find class org.jboss.as.messaging.HornetQLoggerFactory at org.hornetq.utils.ClassloadingUtil$1.run(ClassloadingUtil.java:42) [hornetq-core-2.2.7.Final.jar:] at java.security.AccessController.doPrivileged(Native Method) [:1.6.0_22] at org.hornetq.utils.ClassloadingUtil.safeInitNewInstance(ClassloadingUtil.java:16) [hornetq-core-2.2.7.Final.jar:] at org.hornetq.core.server.impl.HornetQServerImpl.instantiateInstance(HornetQServerImpl.java:1868) [hornetq-core-2.2.7.Final.jar:] at org.hornetq.core.server.impl.HornetQServerImpl.initialiseLogging(HornetQServerImpl.java:1301) [hornetq-core-2.2.7.Final.jar:] at org.hornetq.core.server.impl.HornetQServerImpl.start(HornetQServerImpl.java:541) [hornetq-core-2.2.7.Final.jar:] at org.hornetq.core.server.impl.HornetQServerImpl$SharedStoreBackupActivation$1FailbackChecker$1.run(HornetQServerImpl.java:430) [hornetq-core-2.2.7.Final.jar:] at java.lang.Thread.run(Thread.java:679) [:1.6.0_22] {code} From live server: {code} 12:07:51,993 INFO [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:/topic/test 12:07:52,007 INFO [org.jboss.as] (Controller Boot Thread) JBoss EAP 6.0.0.Alpha2 (AS 7.1.0.CR1-redhat-1) started in 31092ms - Started 155 of 263 services (103 services are passive or on-demand) 12:07:53,292 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Trying reconnection attempt 1 12:07:53,292 DEBUG [org.hornetq.core.remoting.impl.netty.NettyConnector] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Started Netty Connector version 3.2.3.Final-r${buildNumber} 12:07:53,292 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Trying to connect at the main server using connector :org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5446&host=192-168-10-4 12:07:53,293 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Main server is not up. Hopefully there's a backup configured now! 12:07:55,293 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Trying reconnection attempt 2 12:07:55,293 DEBUG [org.hornetq.core.remoting.impl.netty.NettyConnector] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Started Netty Connector version 3.2.3.Final-r${buildNumber} 12:07:55,293 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Trying to connect at the main server using connector :org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5446&host=192-168-10-4 12:07:55,294 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Main server is not up. Hopefully there's a backup {code} I'm not fully sure but it looks like that backup server did not manage to get to "waiting for live to fail" state and live server is not able to detect backup. There is set "DEBUG" level logging for "org.hornetq" in attached logs. I've attached reproducer.zip - steps to use: 1. Download and unzip "reproducer.zip" 2. Prepare live and backup server - "sh prepare.sh" 3. Start live - "sh start-server1.sh server1_hostname" 4. Start backup - "sh start-server2.sh server2_hostname" 5. Kill live server using "kill -9 server1_process_id" 6. Start live server again - "sh start-server1.sh server1_hostname" Note: In reproducer.zip are configuration files standalone-ha-A.xml, standalone-ha-B.xml (A for live, B for backup).