[jboss-dev] Shutting down the app server and removing components
Paul Ferraro
paul.ferraro at redhat.com
Fri Feb 20 13:49:49 EST 2009
Noted.
https://jira.jboss.org/jira/browse/JBAS-6534
On Fri, 2009-02-20 at 09:59 -0600, Brian Stansberry wrote:
> Manik Surtani wrote:
> >
>
> <snip/>
>
> >>> Guys
> >>> While running some basic state transfer tests on AS 5.0, we've seen
> >>> some stack traces when shutting down a server
> >>>
> >>> http://hudson.qa.jboss.com/hudson/view/EAP%20Perf+Fail/job/as5-random-failover-http-buddy-async-jvmkill/10/console-perf01/
> >>>
> >>
>
> <snip/>
> > pertinent log and stack trace
> > snips below:
> >
>
> I believe the issue here is when the AjpProtocol logs the "Stopping
> Coyote AJP/1.3" message you see, what it's basically doing is closing
> the ServerSocket from which regular Sockets are accepted. AFAICT if
> there's a it doesn't disrupt or in any way signal any of the threads
> processing existing sockets that have already been accepted. Those
> threads will keep reading requests off the socket as long as there are any.
>
> I don't know if closing the ServerSocket will cause mod_xx to promptly
> mark the worker as unavailable, or if it waits until 1) it needs to
> create a new connection to the worker or 2) one of the existing
> connections has a failure. Either way, in-flight requests are still
> active after the "Stopping AJP/1.3" message is logged.
>
> >
> > [JBoss] 05:47:33,394 INFO [ServerImpl] Shutting down the server,
> > blockingShutdown: false
> > [JBoss] 05:47:33,397 INFO [ServerImpl] Server exit(0) called
> > [JBoss] 05:47:33,398 INFO [ServerImpl] Runtime shutdown hook called,
> > forceHalt: true
> > [JBoss] 05:47:33,406 INFO [Http11Protocol] Pausing Coyote HTTP/1.1 on
> > http-perf01%2F10.16.88.179-8080
> > [JBoss] 05:47:33,407 INFO [Http11Protocol] Stopping Coyote HTTP/1.1 on
> > http-perf01%2F10.16.88.179-8080
> > [JBoss] 05:47:33,408 INFO [AjpProtocol] Pausing Coyote AJP/1.3 on
> > ajp-perf01%2F10.16.88.179-8009
> > [JBoss] 05:47:33,409 INFO [AjpProtocol] Stopping Coyote AJP/1.3 on
> > ajp-perf01%2F10.16.88.179-8009
> > [JBoss] 05:47:33,460 INFO [ConnectionFactory]
> > org.jboss.jms.server.connectionfactory.ConnectionFactory at 1c42860 undeployed
> > [JBoss] 05:47:33,484 INFO [ConnectionFactory]
> > org.jboss.jms.server.connectionfactory.ConnectionFactory at 38f0e7 undeployed
> > [JBoss] 05:47:33,500 INFO [ConnectionFactory]
> > org.jboss.jms.server.connectionfactory.ConnectionFactory at 4e6c9d undeployed
> > [JBoss] 05:47:33,530 INFO [TomcatDeployment] undeploy,
> > ctxPath=/web-console, vfsUrl=management/console-mgr.sar/web-console.war
> > [JBoss] 05:47:33,584 INFO [QueueService] Queue[/queue/ExpiryQueue] stopped
> > [JBoss] 05:47:33,585 INFO [QueueService] Queue[/queue/DLQ] stopped
> > [JBoss] 05:47:33,595 INFO [QuartzScheduler] Scheduler
> > JBossEJB3QuartzScheduler_$_NON_CLUSTERED shutting down.
> > [JBoss] 05:47:33,595 INFO [QuartzScheduler] Scheduler
> > JBossEJB3QuartzScheduler_$_NON_CLUSTERED paused.
> > [JBoss] 05:47:33,596 INFO [QuartzScheduler] Scheduler
> > JBossEJB3QuartzScheduler_$_NON_CLUSTERED shutdown complete.
> > [JBoss] 05:47:34,645 INFO [ConnectionFactoryBindingService] Unbound
> > ConnectionManager
> > 'jboss.jca:service=ConnectionFactoryBinding,name=JmsXA' from JNDI name
> > 'java:JmsXA'
> > [JBoss] 05:47:34,658 INFO [ServerPeer] JMS ServerPeer[0] stopped
> > [JBoss] 05:47:34,726 INFO [TomcatDeployment] undeploy,
> > ctxPath=/invoker, vfsUrl=httpha-invoker.sar/invoker.war
> > [JBoss] 05:47:34,799 INFO [HttpReplTestPartition] Stopping partition
> > HttpReplTestPartition
> > [JBoss] 05:47:34,802 INFO [RPCManagerImpl] Disconnecting and closing
> > the Channel
> > [JBoss] 05:47:34,874 INFO [RPCManagerImpl] Stopping the RpcDispatcher
> > [JBoss] 05:47:34,888 INFO [HttpReplTestPartition] Partition
> > HttpReplTestPartition stopped.
> > [JBoss] 05:47:34,891 INFO [HttpReplTestPartition] Partition
> > HttpReplTestPartition destroyed.
> > [JBoss] 05:47:34,893 INFO [TomcatDeployment] undeploy, ctxPath=/st,
> > vfsUrl=SessionTest.war
> > [JBoss] 05:47:34,931 INFO [RPCManagerImpl] Disconnecting and closing
> > the Channel
> > [JBoss] 05:47:35,002 INFO [RPCManagerImpl] Stopping the RpcDispatcher
> > [JBoss] 05:47:35,451 INFO [TomcatDeployment] undeploy,
> > ctxPath=/jbossws, vfsUrl=jbossws.sar/jbossws-management.war
> > [JBoss] 05:47:35,454 INFO [TomcatDeployment] undeploy, ctxPath=/juddi,
> > vfsUrl=juddi-service.sar/juddi.war
> > [JBoss] 05:47:35,455 INFO [RegistryServlet] jUDDI Stopping: Cleaning up
> > existing resources.
> > [JBoss] 05:47:35,456 INFO [TomcatDeployment] undeploy, ctxPath=/,
> > vfsUrl=ROOT.war
> > [JBoss] 05:47:35,458 INFO [TomcatDeployment] undeploy,
> > ctxPath=/jmx-console, vfsUrl=jmx-console.war
> > [JBoss] 05:47:35,462 INFO [StandardService] Stopping service jboss.web
> > [JBoss] 05:47:35,598 INFO [MailService] Mail service 'java:/Mail'
> > removed from JNDI
> > [JBoss] 05:47:35,632 INFO [SnmpAgentService] SNMP agent stopped
> > [JBoss] 05:47:35,811 INFO [ConnectionFactoryBindingService] Unbound
> > ConnectionManager 'jboss.jca:service=DataSourceBinding,name=DefaultDS'
> > from JNDI name 'java:DefaultDS'
> > [JBoss] 05:47:35,841 INFO [QuartzScheduler] Scheduler
> > DefaultQuartzScheduler_$_NON_CLUSTERED shutting down.
> > [JBoss] 05:47:35,841 INFO [QuartzScheduler] Scheduler
> > DefaultQuartzScheduler_$_NON_CLUSTERED paused.
> > [JBoss] 05:47:35,845 INFO [SimpleThreadPool] There are still 45 worker
> > threads active. See javadoc runInThread(Runnable) for a possible
> > explanation
> > [JBoss] 05:47:35,845 INFO [QuartzScheduler] Scheduler
> > DefaultQuartzScheduler_$_NON_CLUSTERED shutdown complete.
> > [JBoss] 05:47:35,957 INFO [TransactionManagerService] Stopping
> > transaction recovery manager
> > [JBoss] 05:47:36,213 INFO [HypersonicDatabase] Database standalone
> > closed clean
> > [JBoss] 05:47:36,602 ERROR [CoyoteAdapter] An exception or error
> > occurred in the container during the request processing
> > [JBoss] java.lang.RuntimeException: DistributedCacheManager is null.
> > [JBoss] at
> > org.jboss.web.tomcat.service.session.ClusteredSession.establishDistributedCacheManager(ClusteredSession.java:1802)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.service.session.ClusteredSession.setManager(ClusteredSession.java:506)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.service.session.ClusteredSession.<init>(ClusteredSession.java:399)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.service.session.SessionBasedClusteredSession.<init>(SessionBasedClusteredSession.java:56)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.service.session.JBossCacheManager.createEmptyClusteredSession(JBossCacheManager.java:1410)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.service.session.JBossCacheManager.loadSession(JBossCacheManager.java:1491)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.service.session.JBossCacheManager.findSession(JBossCacheManager.java:462)
> >
> > [JBoss] at
> > org.apache.catalina.connector.Request.doGetSession(Request.java:2322)
> > [JBoss] at
> > org.apache.catalina.connector.Request.getSession(Request.java:2096)
> > [JBoss] at
> > org.jboss.web.tomcat.service.session.JvmRouteValve.checkJvmRoute(JvmRouteValve.java:94)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.service.session.JvmRouteValve.invoke(JvmRouteValve.java:84)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
> >
> > [JBoss] at
> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> >
> > [JBoss] at
> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> >
> > [JBoss] at
> > org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
> >
> > [JBoss] at
> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> >
> > [JBoss] at
> > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
> > [JBoss] at
> > org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
> > [JBoss] at
> > org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
> >
> > [JBoss] at
> > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> > [JBoss] at java.lang.Thread.run(Thread.java:619)
> >
> >
> >>
> >>
> >>> I haven't dug into this in much detail, but it seems as though the
> >>> CoyoteAdapter is still accepting HTTP requests even after the webapps
> >>> and other clustering components have been stopped/removed? Is this a
> >>> known bug in AS 5.0?
> >>> Thoughts?
> >>> Cheers
> >>> Manik
> >>> --
> >>> Manik Surtani
> >>> Lead, JBoss Cache
> >>> http://www.jbosscache.org
> >>> manik at jboss.org
> >>> ------------------------------------------------------------------------
> >>> _______________________________________________
> >>> jboss-development mailing list
> >>> jboss-development at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/jboss-development
> >>
> >> --
> >> Galder Zamarreño
> >> Sr. Software Maintenance Engineer
> >> JBoss, a division of Red Hat
> >> _______________________________________________
> >> jboss-development mailing list
> >> jboss-development at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/jboss-development
> >
> > --
> > Manik Surtani
> > Lead, JBoss Cache
> > http://www.jbosscache.org
> > manik at jboss.org
> >
> >
> >
> >
> >
> > _______________________________________________
> > jboss-development mailing list
> > jboss-development at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/jboss-development
>
>
More information about the jboss-development
mailing list