[jboss-jira] [JBoss JIRA] (AS7-6711) Unable to restart server group in domain mode

Mon Mar 11 22:38:41 EDT 2013

    [ https://issues.jboss.org/browse/AS7-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760434#comment-12760434 ] 

Brian Stansberry commented on AS7-6711:
---------------------------------------

It's simpler than the original description. Multiple servers aren't required, nor is restart-servers required. Calling /server-group=x:start-servers when the master HC has just a single server in group "x" will result in the problem.

The problem is that:

1) OperationCoordinatorStepHandler calls context.acquireControllerLock() at line 209. The lock will be held until the server connects.
2) The registering server needs to acquire the lock as part of registration, ServerToHostProtocolHandler line 254. Until it gets the lock, it cannot connect.

This deadlocks.

A likely workaround is to not run servers on the master HostController. The lock acquired in 1) is done on the master.

> Unable to restart server group in domain mode
> ---------------------------------------------
>
>                 Key: AS7-6711
>                 URL: https://issues.jboss.org/browse/AS7-6711
>             Project: Application Server 7
>          Issue Type: Bug
>          Components: Domain Management
>    Affects Versions: EAP 6.1.0.Alpha (7.2.0.Final)
>            Reporter: Brian Stansberry
>            Assignee: Brian Stansberry
>            Priority: Critical
>             Fix For: 8.0.0.Alpha1
>
>         Attachments: AS7-6711-threads.txt
>
>
> Restarting the servers in the standard domain.xml main-server-group will result in a hang in the CLI and in the master HC.
> To reproduce start the standard domain.sh and in another window launch the CLI and:
> [domain at localhost:9999 /] /server-group=main-server-group:restart-servers
> Resulting console log output:
> [Host Controller] 18:17:30,511 INFO  [org.jboss.as.host.controller] (management-handler-thread - 3) JBAS010923: Stopping server server-one
> 18:17:30,515 INFO  [org.jboss.as.process.Server:server-one.status] (ProcessController-threads - 5) JBAS012018: Stopping process 'Server:server-one'
> [Server:server-one] 18:17:30,599 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-4) JBAS010410: Unbound JCA ConnectionFactory [java:/JmsXA]
> [Server:server-one] 18:17:30,600 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) JBAS010409: Unbound data source [java:jboss/datasources/ExampleDS]
> [Server:server-one] 18:17:30,611 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003075: Coyote HTTP/1.1 pausing on: http-localhost/127.0.0.1:8080
> [Server:server-one] 18:17:30,612 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003077: Coyote HTTP/1.1 stopping on : http-localhost/127.0.0.1:8080
> [Server:server-one] 18:17:30,616 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 79) JBAS011605: Unbound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory
> [Server:server-one] 18:17:30,615 INFO  [org.apache.catalina.core] (MSC service thread 1-2) JBWEB001079: Container org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/] has not been started
> [Server:server-one] 18:17:30,640 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-3) JBAS010418: Stopped Driver service with driver-name = h2
> [Server:server-one] 18:17:30,664 INFO  [org.hornetq.ra] (MSC service thread 1-4) HQ151005: HornetQ resource adaptor stopped
> [Server:server-one] 18:17:30,704 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221004: HornetQ Server version 2.3.0.CR1 (buzzzzz!, 122) [93f152f7-8aa1-11e2-8c0e-99b598ec3b97] stopped
> [Host Controller] 18:17:30,722 INFO  [org.jboss.as.host.controller] (Remoting "pingguo.local:MANAGEMENT" read-1) JBAS010926: Unregistering server server-one
> [Server:server-one] 18:17:30,754 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015950: JBoss AS 8.0.0.Alpha1-SNAPSHOT "TBD" stopped in 189ms
> [Server:server-one] 
> 18:17:30,815 INFO  [org.jboss.as.process.Server:server-one.status] (reaper for Server:server-one) JBAS012010: Process 'Server:server-one' finished with an exit status of 0
> [Host Controller] 18:17:30,818 INFO  [org.jboss.as.host.controller] (ProcessControllerConnection-thread - 2) JBAS010926: Unregistering server server-one
> [Host Controller] 18:17:30,836 INFO  [org.jboss.as.host.controller] (management-handler-thread - 3) JBAS010922: Starting server server-one
> 18:17:30,845 INFO  [org.jboss.as.process.Server:server-one.status] (ProcessController-threads - 5) JBAS012017: Starting process 'Server:server-one'
> [Server:server-one] 18:17:31,676 INFO  [org.jboss.modules] (main) JBoss Modules version 1.2.0.CR2
> [Server:server-one] 18:17:32,178 INFO  [org.jboss.msc] (main) JBoss MSC version 1.1.1.Final
> [Server:server-one] 18:17:32,332 INFO  [org.jboss.as] (MSC service thread 1-4) JBAS015899: JBoss AS 8.0.0.Alpha1-SNAPSHOT "TBD" starting
> [Server:server-one] 18:17:32,530 INFO  [org.xnio] (MSC service thread 1-3) XNIO Version 3.1.0.Beta9
> [Server:server-one] 18:17:32,538 INFO  [org.xnio.nio] (MSC service thread 1-3) XNIO NIO Implementation Version 3.1.0.Beta9
> [Server:server-one] 18:17:32,559 INFO  [org.jboss.remoting] (MSC service thread 1-3) JBoss Remoting version 3.2.15.GA
> [Host Controller] 18:17:33,607 INFO  [org.jboss.as.domain.controller.mgmt] (Remoting "pingguo.local:MANAGEMENT" task-16) JBAS010920: Server [Server:server-one] registered using connection [Channel ID 2bacdeb7 (inbound) of Remoting connection 49660c5f to /127.0.0.1:60647]
> After that everything hangs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira