8:48 a.m.
Adrian,
Today the AS testsuite runs on hudson are starting to fail
intermittently[1] due to the jboss-minimal-tests target's shutdown.sar
not stopping the AS. There seems to be a classloading problem when
ExitOnShutdown's anonymous Runnable tries to execute.[2]
I looked at a successful run going back a month and the WARN in the
logging below was there, so that's not new.
JIRA for this is https://jira.jboss.org/jira/browse/JBAS-7688
[1] https://hudson.jboss.org/hudson/job/JBoss-AS-6.0.x-testSuite-sun16/537/
https://hudson.jboss.org/hudson/job/JBoss-AS-6.0.x-testSuite-sun16/539/
[2]
2010-02-02 04:58:17,623 DEBUG
[org.jboss.test.jmx.shutdown.ExitOnShutdown] (HDScanner) Stopping
jboss.test:name=ExitOnShutdown
2010-02-02 04:58:17,624 WARN
[org.jboss.virtual.plugins.context.AbstractVirtualFileHandler]
(HDScanner) No such existing handler, falling back to old root + path:
vfszip:/mnt/hudson_workspace/workspace/JBoss-AS-6.0.x-testSuite-sun16/JBossAS/build/target/jboss-6.0.0-SNAPSHOT/server/minimal/deploy/shutdown.sar/org/jboss/test/jmx/shutdown/ExitOnShutdown$1.class
2010-02-02 04:58:17,636 DEBUG
[org.jboss.test.jmx.shutdown.ExitOnShutdown] (HDScanner) Stopped
jboss.test:name=ExitOnShutdown
2010-02-02 04:58:17,636 DEBUG [org.jboss.system.ServiceController]
(HDScanner) destroying service: jboss.test:name=ExitOnShutdown
2010-02-02 04:58:17,637 DEBUG [org.jboss.system.ServiceController]
(HDScanner) removing service: jboss.test:name=ExitOnShutdown
2010-02-02 04:58:17,637 DEBUG [org.jboss.system.ServiceCreator]
(HDScanner) Removing mbean from server: jboss.test:name=ExitOnShutdown
2010-02-02 04:58:17,642 DEBUG
[org.jboss.deployers.structure.spi.helpers.AbstractDeploymentContext]
(HDScanner) Removed component jboss.test:name=ExitOnShutdown from
vfszip:/mnt/hudson_workspace/workspace/JBoss-AS-6.0.x-testSuite-sun16/JBossAS/build/target/jboss-6.0.0-SNAPSHOT/server/minimal/deploy/shutdown.sar/
2010-02-02 04:58:17,647 DEBUG
[org.jboss.aop.asintegration.jboss5.RegisterModuleCallback] (HDScanner)
Removing module VFSDeploymentClassLoaderPolicyModule shutdown.sar:0.0.0
2010-02-02 04:58:17,647 DEBUG
[org.jboss.classloader.spi.base.BaseClassLoaderDomain] (HDScanner)
ClassLoaderDomain@1f8bd0d{DefaultDomain} unregisterClassLoader
BaseClassLoader@8c1852{vfszip:/mnt/hudson_workspace/workspace/JBoss-AS-6.0.x-testSuite-sun16/JBossAS/build/target/jboss-6.0.0-SNAPSHOT/server/minimal/deploy/shutdown.sar/}
2010-02-02 04:58:17,648 DEBUG
[org.jboss.mx.loading.UnifiedLoaderRepository3] (HDScanner)
UnifiedLoaderRepository removed(false) null
2010-02-02 04:58:17,649 DEBUG
[org.jboss.classloader.spi.base.BaseClassLoaderPolicy] (HDScanner)
VFSClassLoaderPolicy@190ed52{vfszip:/mnt/hudson_workspace/workspace/JBoss-AS-6.0.x-testSuite-sun16/JBossAS/build/target/jboss-6.0.0-SNAPSHOT/server/minimal/deploy/shutdown.sar/}
shutdown!
2010-02-02 04:58:17,649 DEBUG
[org.jboss.classloader.spi.base.BaseClassLoader] (HDScanner)
BaseClassLoader@8c1852{vfszip:/mnt/hudson_workspace/workspace/JBoss-AS-6.0.x-testSuite-sun16/JBossAS/build/target/jboss-6.0.0-SNAPSHOT/server/minimal/deploy/shutdown.sar/}
shutdown!
2010-02-02 04:58:17,650 DEBUG
[org.jboss.classloading.spi.dependency.Domain] (HDScanner)
org.jboss.classloading.spi.dependency.Domain@3a6e5c{DefaultDomain}
remove module VFSDeploymentClassLoaderPolicyModule shutdown.sar:0.0.0
2010-02-02 04:58:17,650 DEBUG
[org.jboss.deployers.vfs.plugins.classloader.InMemoryClassesDeployer]
(HDScanner) Removing dynamic class root for
vfszip:/mnt/hudson_workspace/workspace/JBoss-AS-6.0.x-testSuite-sun16/JBossAS/build/target/jboss-6.0.0-SNAPSHOT/server/minimal/deploy/shutdown.sar/
2010-02-02 04:58:17,658 DEBUG
[org.jboss.deployers.plugins.deployers.DeployersImpl] (HDScanner) Fully
Undeployed
vfszip:/mnt/hudson_workspace/workspace/JBoss-AS-6.0.x-testSuite-sun16/JBossAS/build/target/jboss-6.0.0-SNAPSHOT/server/minimal/deploy/shutdown.sar/
2010-02-02 04:58:17,674 ERROR [STDERR] (ExitOnShutdown) Exception in
thread "ExitOnShutdown" java.lang.NoClassDefFoundError: java/lang/System
2010-02-02 04:58:17,675 ERROR [STDERR] (ExitOnShutdown) at
org.jboss.test.jmx.shutdown.ExitOnShutdown$1.run(ExitOnShutdown.java:52)
2010-02-02 04:58:17,676 ERROR [STDERR] (ExitOnShutdown) at
java.lang.Thread.run(Thread.java:619)
2010-02-02 04:58:17,676 ERROR [STDERR] (ExitOnShutdown) Caused by:
java.lang.ClassNotFoundException: Class not found java.lang.System
2010-02-02 04:58:17,676 ERROR [STDERR] (ExitOnShutdown) at
org.jboss.classloader.spi.base.BaseClassLoader.loadClassFromDomain(BaseClassLoader.java:892)
2010-02-02 04:58:17,677 ERROR [STDERR] (ExitOnShutdown) at
org.jboss.classloader.spi.base.BaseClassLoader.doLoadClass(BaseClassLoader.java:522)
2010-02-02 04:58:17,677 ERROR [STDERR] (ExitOnShutdown) at
org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:467)
2010-02-02 04:58:17,677 ERROR [STDERR] (ExitOnShutdown) at
java.lang.ClassLoader.loadClass(ClassLoader.java:252)
2010-02-02 04:58:17,678 ERROR [STDERR] (ExitOnShutdown) at
java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
2010-02-02 04:58:17,678 ERROR [STDERR] (ExitOnShutdown) ... 2 more
2010-02-02 04:59:47,314 INFO
[org.jboss.bootstrap.impl.base.server.AbstractServer] (Thread-5)
Stopping: JBossAS [6.0.0.SNAPSHOT (build: SVNTag=JBoss_6.0.0-SNAPSHOT
date=r100244)]
2010-02-02 04:59:47,314 DEBUG
[org.jboss.bootstrap.impl.as.lifecycle.AbstractKernelEventLifecycleEventHandler]
(Thread-5) Fired:
AbstractEvent@a801b0{source=org.jboss.kernel.plugins.event.AbstractEventManager(a)1fe0816
type=org.jboss.system.server.stopped seq=0 time=1265104787314
context=1265104787314}
2010-02-02 04:59:47,314 DEBUG
[org.jboss.bootstrap.impl.base.server.AbstractServer] (Thread-5) Sent
JMX Notification: org.jboss.system.server.stopped
--
Brian Stansberry
Lead, AS Clustering
JBoss by Red Hat
8:54 a.m.
On 02/02/2010 05:48 PM, Brian Stansberry wrote:
Adrian,
Today the AS testsuite runs on hudson are starting to fail
intermittently[1] due to the jboss-minimal-tests target's shutdown.sar
not stopping the AS. There seems to be a classloading problem when
ExitOnShutdown's anonymous Runnable tries to execute.[2]
I hypothesize that the classloader is (sometimes) being destroyed before
the thread gets going, causing it to be unable to load anything at all.
Two problems with this are:
1. The exit thread needs a functional classloader to get ahold of
java.lang.System
2. Regardless of the status of the classloader, java.* should always be
available (shouldn't it?)
To work around (1) I committed a patch to ExitOnShutdown which uses a
CountDownLatch to suspend the stopService() method for just long enough to
get ahold of a Runtime instance which can have .exit() called on it, thus
(hopefully) preventing the classloader from disappearing before that time.
So to duplicate this issue you'll have to revert that locally (rev 100309).
For (2), I leave it open for discussion I guess :)
- DML
9:04 a.m.
David Lloyd just committed a patch to ExitOnShutdown to try to work
around this (thanks, David). It attempts to sweep the problem under the
rug so the testsuite can run. Said sweeping is OK, since there's a
critical JIRA highlighting the problem.
On 02/02/2010 05:48 PM, Brian Stansberry wrote:
Adrian,
Today the AS testsuite runs on hudson are starting to fail
intermittently[1] due to the jboss-minimal-tests target's shutdown.sar
not stopping the AS. There seems to be a classloading problem when
ExitOnShutdown's anonymous Runnable tries to execute.[2]
I looked at a successful run going back a month and the WARN in the
logging below was there, so that's not new.
JIRA for this is https://jira.jboss.org/jira/browse/JBAS-7688
--
Brian Stansberry
Lead, AS Clustering
JBoss by Red Hat
5207
days inactive
5208
days old
jboss-development@lists.jboss.org
2 comments
2 participants
participants (2)
-
Brian Stansberry -
David M. Lloyd