[jboss-jira] [JBoss JIRA] (AS7-5163) OSGi bootstrap may exhaust bootstrap threads and deadlock

Fri Jul 13 04:12:13 EDT 2012

    [ https://issues.jboss.org/browse/AS7-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705825#comment-12705825 ] 

Thomas Diesler edited comment on AS7-5163 at 7/13/12 4:12 AM:
--------------------------------------------------------------

Alexander says:

{quote}
The problem can be reproduced with 100% success rate by setting the number of deployment threads to 1. This can be achieved using the <option value="-Dorg.jboss.server.bootstrap.maxThreads=1"/> jvm property.

I think the root of the problem is in org.jboss.as.server.moduleservice.ServiceModuleLoader and the asynchronous execution of listener tasks. The method findModule looks like this:

        ModuleSpecLoadListener listener = new ModuleSpecLoadListener(identifier);
        controller.addListener(listener);
        return listener.getModuleSpec();

And the successfull, not-timeouted completion of listener.getModuleSpec(); requires that the listener task added from here - controller.addListener(listener) - to be completed successfully. Unfortunately, this won't happen since there is only 1 deployer thread, which is currently busy. So, after 2 seconds, the ModuleTimeoutException is thrown.

Of course, running 2 or more threads means that the problem will occur much less often, but still happens sometimes.
Also, I've noted that it only appears with osgi bundles - haven't seen it with enterprise archives or jboss modules.
{quote}

      was (Author: thomas.diesler):
    Alexander says:

The problem can be reproduced with 100% success rate by setting the number of deployment threads to 1. This can be achieved using the <option value="-Dorg.jboss.server.bootstrap.maxThreads=1"/> jvm property.

I think the root of the problem is in org.jboss.as.server.moduleservice.ServiceModuleLoader and the asynchronous execution of listener tasks. The method findModule looks like this:

    ...

        ModuleSpecLoadListener listener = new ModuleSpecLoadListener(identifier);

        controller.addListener(listener);

        return listener.getModuleSpec();

    ...

And the successfull, not-timeouted completion of listener.getModuleSpec(); requires that the listener task added from here - controller.addListener(listener) - to be completed successfully. Unfortunately, this won't happen since there is only 1 deployer thread, which is currently busy. So, after 2 seconds, the ModuleTimeoutException is thrown.

Of course, running 2 or more threads means that the problem will occur much less often, but still happens sometimes.

Also, I've noted that it only appears with osgi bundles - haven't seen it with enterprise archives or jboss modules.

> OSGi bootstrap may exhaust bootstrap threads and deadlock
> ---------------------------------------------------------
>
>                 Key: AS7-5163
>                 URL: https://issues.jboss.org/browse/AS7-5163
>             Project: Application Server 7
>          Issue Type: Bug
>          Components: OSGi
>    Affects Versions: 7.1.1.Final
>            Reporter: David Bosschaert
>            Assignee: Thomas Diesler
>            Priority: Critical
>             Fix For: 7.1.3.Final (EAP), 7.2.0.Alpha1
>
>
> Reported on the Forum:
> {code}org.jboss.msc.service.fail JB${bisas.pid} MSC service thread 1-16 ERROR MSC00001: Failed to start service jbosgi.AutoInstallProvider.COMPLETE: org.jboss.msc.service.StartException in service jbosgi.AutoInstallProvider.COMPLETE: Failed to start service
>  at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.IllegalStateException: Cannot load module: deployment.osgi.enterprise:4.2.0.201003190513
>  at org.jboss.osgi.framework.internal.ResolverPlugin.loadModules(ResolverPlugin.java:261)
>  at org.jboss.osgi.framework.internal.ResolverPlugin.applyResolverResults(ResolverPlugin.java:229)
>  at org.jboss.osgi.framework.internal.ResolverPlugin.resolveAll(ResolverPlugin.java:209)
>  at org.jboss.osgi.framework.internal.PackageAdminPlugin.resolveBundles(PackageAdminPlugin.java:385)
>  at org.jboss.as.osgi.service.AutoInstallIntegration$1.start(AutoInstallIntegration.java:176)
>  at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811)
>  at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746)
>  ... 3 more
> Caused by: org.jboss.modules.ModuleLoadException: Timeout waiting for module service: deployment.osgi.enterprise:4.2.0.201003190513
>  at org.jboss.as.server.moduleservice.ServiceModuleLoader$ModuleSpecLoadListener.getModuleSpec(ServiceModuleLoader.java:133)
>  at org.jboss.as.server.moduleservice.ServiceModuleLoader.findModule(ServiceModuleLoader.java:174)
>  at org.jboss.modules.ModuleLoader.loadModuleLocal(ModuleLoader.java:275)
>  at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:222)
>  at org.jboss.as.server.moduleservice.ServiceModuleLoader.preloadModule(ServiceModuleLoader.java:158)
>  at org.jboss.modules.ModuleLoader.preloadExportedModule(ModuleLoader.java:233)
>  at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:246)
>  at org.jboss.as.osgi.service.ModuleLoaderIntegration.preloadModule(ModuleLoaderIntegration.java:191)
>  at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:204)
>  at org.jboss.osgi.framework.internal.ModuleManagerPlugin.loadModule(ModuleManagerPlugin.java:556)
>  at org.jboss.osgi.framework.internal.ResolverPlugin.loadModules(ResolverPlugin.java:259)
>  ... 9 more{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira