[mod_cluster-issues] [JBoss JIRA] (MODCLUSTER-570) Multiple invalid nodes created on graceful httpd restart
Antoine Cotten (JIRA)
issues at jboss.org
Thu Mar 30 11:29:00 EDT 2017
[ https://issues.jboss.org/browse/MODCLUSTER-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13386589#comment-13386589 ]
Antoine Cotten edited comment on MODCLUSTER-570 at 3/30/17 11:28 AM:
---------------------------------------------------------------------
I managed to reproduce on 1.3.6.Final. It's actually as simple as doing multiple graceful restarts in a row, no need for an application server (but happens with apps servers as well).
And most importantly, it seems specific to *Alpine Linux*.
What I did:
# Run httpd with mod_cluster
{code}docker run --name mc -p 8080:80 -p 7777:6666 antoineco/httpd-mod_cluster:1.3.6-alpine{code}
# Uncomment the mod_cluster modules in the config, and uncomment the very last line ({{Include}}) to enable the manager vhost
{code}docker exec -it mc vi conf/httpd.conf{code}
# Gracefully reload httpd:
{code}docker exec mc apachectl -k graceful{code}
# Repeat 3. two times if no app server got registered (happens right away if an app server is registered)
The same error messages as described in the description will surface.
was (Author: antoineco):
I managed to reproduce on 1.3.6.Final. It's actually as simple as doing multiple graceful restarts in a row, no need for an application server.
What I did:
# Run httpd with mod_cluster
{code}docker run --name mc -p 8080:80 -p 7777:6666 antoineco/httpd-mod_cluster:1.3.6-alpine{code}
# Uncomment the mod_cluster modules in the config, and uncomment the very last line ({{Include}}) to enable the manager vhost
{code}docker exec -it mc vi conf/httpd.conf{code}
# Gracefully reload httpd:
{code}docker exec mc apachectl -k graceful{code}
# Repeat 3. two times
The same error messages as described in the description will surface.
This seems to happen only the first time mod_cluster is started, if I stop httpd and start it again later on I can't reproduce.
> Multiple invalid nodes created on graceful httpd restart
> --------------------------------------------------------
>
> Key: MODCLUSTER-570
> URL: https://issues.jboss.org/browse/MODCLUSTER-570
> Project: mod_cluster
> Issue Type: Bug
> Components: Native (httpd modules)
> Affects Versions: 1.3.5.Final, 1.3.6.Final
> Environment: httpd 2.4.25
> Reporter: Antoine Cotten
> Assignee: Jean-Frederic Clere
> Priority: Minor
> Attachments: mod_cluster_after_graceful.png, mod_cluster_before_graceful.png
>
>
> mod_cluster creates a bunch of invalid nodes after a graceful restart of Apache httpd ({{SIGUSR1}}, {{SIGWINCH}}). The issue happens with or without backend servers in the pool.
> httpd logs:
> {code}
> [mpm_event:notice] [pid 1:tid 140313388616520] AH00493: SIGUSR1 received. Doing graceful restart
> [:notice] [pid 1:tid 140313388616520] Advertise initialized for process 1
> [mpm_event:notice] [pid 1:tid 140313388616520] AH00489: Apache/2.4.25 (Unix) mod_cluster/1.3.5.Final configured -- resuming normal operations
> [core:notice] [pid 1:tid 140313388616520] AH00094: Command line: 'httpd -D FOREGROUND'
> [:notice] [pid 116:tid 140313388616520] Created: worker for ://: failed: Unable to parse URL
> [:notice] [pid 116:tid 140313388616520] Created: worker for ://: failed: Unable to parse URL
> [:notice] [pid 116:tid 140313388616520] Created: worker for ://: failed: Unable to parse URL
> # ... trimmed 48 lines ...
> [:notice] [pid 145:tid 140313388616520] Created: worker for ://: failed: Unable to parse URL
> [:notice] [pid 145:tid 140313388616520] Created: worker for ://: failed: Unable to parse URL
> [:notice] [pid 145:tid 140313388616520] Created: worker for ://: failed: Unable to parse URL
> [:notice] [pid 145:tid 140313388616520] Created: worker for ://:\x9d\x7f failed: Unable to parse URL
> [:notice] [pid 145:tid 140313388616520] add_balancer_node: balancer safe-name (balancer://4:6666\r\nX-Manager-Url: /c8dae916-7642-4deb-b841-2f3435613749\r\nX-Manager-Protocol: http\r\nX-Manager-Host: 172.17.0.4\r\n\r\n) too long
> {code}
> Application server logs (tomcat8):
> {code}
> Feb 01, 2017 11:34:25 AM org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler sendRequest
> ERROR: MODCLUSTER000042: Error MEM sending STATUS command to 172.17.0.4/172.17.0.4:6666, configuration will be reset: MEM: Can't read node with "java1-depl1-1234" JVMRoute
> Feb 01, 2017 11:34:35 AM org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor processChildren
> SEVERE: Exception invoking periodic operation:
> java.lang.IllegalArgumentException: Node: [1],Name: ,Balancer: ,LBGroup: ,Host: ,Port: ,Type: ,Flushpackets: Off,Flushwait: 0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 0
> Node: [2],Name: ,Balancer: ,LBGroup: ,Host: ,Port: ,Type: ,Flushpackets: Off,Flushwait: 0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 0
> Node: [3],Name: ,Balancer: ,LBGroup: ,Host: ,Port: ,Type: ,Flushpackets: Off,Flushwait: 0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 0
> # ... trimmed lines ...
> Node: [18],Name: ,Balancer: ,LBGroup: ,Host: ,Port: ,Type: ,Flushpackets: Off,Flushwait: 0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 0
> Node: [19],Name: ,Balancer: ,LBGroup: ,Host: ,Port: ,Type: ,Flushpackets: Off,Flushwait: 0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 0
> Vhost: [0:0:1], Alias:
> Vhost: [0:0:2], Alias:
> Vhost: [0:0:3], Alias:
> # ... trimmed lines ...
> Vhost: [0:0:19], Alias:
> Vhost: [0:0:20], Alias:
> Context: [0:0:1], Context: , Status: REMOVED
> Context: [0:0:2], Context: , Status: REMOVED
> Context: [0:0:3], Context: , Status: REMOVED
> # ... trimmed lines ...
> Context: [0:0:97], Context: , Status: REMOVED
> Context: [0:0:98], Context: , Status: REMOVED
> Context: [0:0:99], Context: , Status: REMOVED
> Context: [0:0:1], Context: , Status: REMOVED
> at org.jboss.modcluster.mcmp.impl.DefaultMCMPResponseParser.parseInfoResponse(DefaultMCMPResponseParser.java:96)
> at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.status(DefaultMCMPHandler.java:396)
> at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.status(DefaultMCMPHandler.java:365)
> at org.jboss.modcluster.ModClusterService.status(ModClusterService.java:457)
> at org.jboss.modcluster.container.catalina.CatalinaEventHandlerAdapter.lifecycleEvent(CatalinaEventHandlerAdapter.java:252)
> at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
> at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
> at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1374)
> at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1546)
> at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1524)
> at java.lang.Thread.run(Thread.java:745)
> Feb 01, 2017 11:35:05 AM org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler sendRequest
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
More information about the mod_cluster-issues
mailing list