]
Antoine Cotten edited comment on MODCLUSTER-570 at 3/30/17 9:35 AM:
--------------------------------------------------------------------
I managed to reproduce on 1.3.6.Final. It's actually as simple as doing multiple
graceful restarts in a row, no need for an application server.
What I did:
# Run httpd with mod_cluster
{code}docker run --name mc -p 8080:80 -p 7777:6666
antoineco/httpd-mod_cluster:1.3.6-alpine{code}
# Uncomment the mod_cluster modules in the config, and uncomment the very last line
({{Include}}) to enable the manager vhost
{code}docker exec -it mc vi conf/httpd.conf{code}
# Gracefully reload httpd:
{code}docker exec mc apachectl -k graceful{code}
# Repeat 3. two times
The same error messages as described in the description will surface.
This seems to happen only the first time mod_cluster is started, if I stop httpd and start
it again later on I can't reproduce.
was (Author: antoineco):
I managed to reproduce on 1.3.6.Final. It's actually as simple as doing multiple
graceful restarts in a row, no need for an application server.
What I did:
# Run httpd with mod_cluster
{code}docker run --name mc -p 8080:80 -p 7777:6666
antoineco/httpd-mod_cluster:1.3.6-alpine{code}
# Uncomment the mod_cluster modules in the config, and uncomment the very last line
({{Include}}) to enable the manager vhost
{code}docker exec -it mc vi conf/httpd.conf{code}
# Gracefully reload httpd:
{code}docker exec mc apachectl -k graceful{code}
# Repeat 3. two times
The same error messages as described in the description will surface.
Multiple invalid nodes created on graceful httpd restart
--------------------------------------------------------
Key: MODCLUSTER-570
URL:
https://issues.jboss.org/browse/MODCLUSTER-570
Project: mod_cluster
Issue Type: Bug
Components: Native (httpd modules)
Affects Versions: 1.3.5.Final, 1.3.6.Final
Environment: httpd 2.4.25
Reporter: Antoine Cotten
Assignee: Jean-Frederic Clere
Priority: Minor
Attachments: mod_cluster_after_graceful.png, mod_cluster_before_graceful.png
mod_cluster creates a bunch of invalid nodes after a graceful restart of Apache httpd
({{SIGUSR1}}, {{SIGWINCH}}). The issue happens with or without backend servers in the
pool.
httpd logs:
{code}
[mpm_event:notice] [pid 1:tid 140313388616520] AH00493: SIGUSR1 received. Doing graceful
restart
[:notice] [pid 1:tid 140313388616520] Advertise initialized for process 1
[mpm_event:notice] [pid 1:tid 140313388616520] AH00489: Apache/2.4.25 (Unix)
mod_cluster/1.3.5.Final configured -- resuming normal operations
[core:notice] [pid 1:tid 140313388616520] AH00094: Command line: 'httpd -D
FOREGROUND'
[:notice] [pid 116:tid 140313388616520] Created: worker for ://: failed: Unable to parse
URL
[:notice] [pid 116:tid 140313388616520] Created: worker for ://: failed: Unable to parse
URL
[:notice] [pid 116:tid 140313388616520] Created: worker for ://: failed: Unable to parse
URL
# ... trimmed 48 lines ...
[:notice] [pid 145:tid 140313388616520] Created: worker for ://: failed: Unable to parse
URL
[:notice] [pid 145:tid 140313388616520] Created: worker for ://: failed: Unable to parse
URL
[:notice] [pid 145:tid 140313388616520] Created: worker for ://: failed: Unable to parse
URL
[:notice] [pid 145:tid 140313388616520] Created: worker for ://:\x9d\x7f failed: Unable
to parse URL
[:notice] [pid 145:tid 140313388616520] add_balancer_node: balancer safe-name
(balancer://4:6666\r\nX-Manager-Url:
/c8dae916-7642-4deb-b841-2f3435613749\r\nX-Manager-Protocol: http\r\nX-Manager-Host:
172.17.0.4\r\n\r\n) too long
{code}
Application server logs (tomcat8):
{code}
Feb 01, 2017 11:34:25 AM org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler sendRequest
ERROR: MODCLUSTER000042: Error MEM sending STATUS command to 172.17.0.4/172.17.0.4:6666,
configuration will be reset: MEM: Can't read node with "java1-depl1-1234"
JVMRoute
Feb 01, 2017 11:34:35 AM
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor processChildren
SEVERE: Exception invoking periodic operation:
java.lang.IllegalArgumentException: Node: [1],Name: ,Balancer: ,LBGroup: ,Host: ,Port:
,Type: ,Flushpackets: Off,Flushwait: 0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read:
0,Transfered: 0,Connected: 0,Load: 0
Node: [2],Name: ,Balancer: ,LBGroup: ,Host: ,Port: ,Type: ,Flushpackets: Off,Flushwait:
0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 0
Node: [3],Name: ,Balancer: ,LBGroup: ,Host: ,Port: ,Type: ,Flushpackets: Off,Flushwait:
0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 0
# ... trimmed lines ...
Node: [18],Name: ,Balancer: ,LBGroup: ,Host: ,Port: ,Type: ,Flushpackets: Off,Flushwait:
0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 0
Node: [19],Name: ,Balancer: ,LBGroup: ,Host: ,Port: ,Type: ,Flushpackets: Off,Flushwait:
0,Ping: 0,Smax: 0,Ttl: 0,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 0
Vhost: [0:0:1], Alias:
Vhost: [0:0:2], Alias:
Vhost: [0:0:3], Alias:
# ... trimmed lines ...
Vhost: [0:0:19], Alias:
Vhost: [0:0:20], Alias:
Context: [0:0:1], Context: , Status: REMOVED
Context: [0:0:2], Context: , Status: REMOVED
Context: [0:0:3], Context: , Status: REMOVED
# ... trimmed lines ...
Context: [0:0:97], Context: , Status: REMOVED
Context: [0:0:98], Context: , Status: REMOVED
Context: [0:0:99], Context: , Status: REMOVED
Context: [0:0:1], Context: , Status: REMOVED
at
org.jboss.modcluster.mcmp.impl.DefaultMCMPResponseParser.parseInfoResponse(DefaultMCMPResponseParser.java:96)
at
org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.status(DefaultMCMPHandler.java:396)
at
org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.status(DefaultMCMPHandler.java:365)
at org.jboss.modcluster.ModClusterService.status(ModClusterService.java:457)
at
org.jboss.modcluster.container.catalina.CatalinaEventHandlerAdapter.lifecycleEvent(CatalinaEventHandlerAdapter.java:252)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1374)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1546)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1524)
at java.lang.Thread.run(Thread.java:745)
Feb 01, 2017 11:35:05 AM org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler sendRequest
{code}