mod_cluster segfaults in process_info() due to wrongly generated assembler instruction movslq --------------------------------------------------------------------------------------------- Key: MODCLUSTER-582 URL: https://issues.jboss.org/browse/MODCLUSTER-582 Project: mod_cluster Issue Type: Bug Components: Native (httpd modules) Affects Versions: 1.2.13.Final, 1.3.6.Final, 2.0.0.Alpha1 Environment: RHEL, gcc Reporter: Michal Karm Babacek Assignee: Michal Karm Babacek Priority: Blocker Fix For: 1.3.8.Final, 2.0.0.Alpha1 Affected code: [1.3.x|https://github.com/modcluster/mod_cluster/blob/1.3.x/native/mod_man...] [1.2.x|https://github.com/modcluster/mod_cluster/blob/1.2.x/native/mod_man...] [master|https://github.com/modcluster/mod_proxy_cluster/blob/master/native...] Martin Osvald from Red Hat: {quote} h3. Description of problem: mod_cluster segfaults in process_info() due to wrongly generated assembler instruction movslq How reproducible: intermittent Steps to Reproduce: TBD or N/A h3. Actual results: {code} Core was generated by `/usr/sbin/httpd.worker'. Program terminated with signal 11, Segmentation fault. #0 process_info (r=0x7fc2840386a8, errtype=<value optimized out>) at mod_manager.c:1150 1150 ap_rprintf(r, ",Elected: %d,Read: %d,Transfered: %d,Connected: %d,Load: %d\n", (gdb) bt #0 process_info (r=0x7fc2840386a8, errtype=<value optimized out>) at mod_manager.c:1150 #1 0x00007fc2b9d8344a in manager_handler (r=0x7fc2840386a8) at mod_manager.c:2705 #2 0x00007fc2bf54f640 in ap_run_handler (r=0x7fc2840386a8) at /usr/src/debug/httpd-2.2.15/server/config.c:158 #3 0x00007fc2bf552efe in ap_invoke_handler (r=0x7fc2840386a8) at /usr/src/debug/httpd-2.2.15/server/config.c:376 #4 0x00007fc2bf55e300 in ap_process_request (r=0x7fc2840386a8) at /usr/src/debug/httpd-2.2.15/modules/http/http_request.c:282 #5 0x00007fc2bf55b178 in ap_process_http_connection (c=0x7fc2ac051d00) at /usr/src/debug/httpd-2.2.15/modules/http/http_core.c:190 #6 0x00007fc2bf557148 in ap_run_process_connection (c=0x7fc2ac051d00) at /usr/src/debug/httpd-2.2.15/server/connection.c:43 #7 0x00007fc2bf564052 in process_socket (thd=0x7fc2bffe8d30, dummy=<value optimized out>) at /usr/src/debug/httpd-2.2.15/server/mpm/worker/worker.c:544 #8 worker_thread (thd=0x7fc2bffe8d30, dummy=<value optimized out>) at /usr/src/debug/httpd-2.2.15/server/mpm/worker/worker.c:894 #9 0x00007fc2bddf99d1 in start_thread (arg=0x7fc2a51f4700) at pthread_create.c:301 #10 0x00007fc2bdb46a8d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115 (gdb) {code} h3. Expected results: no segfault h3.Additional info: It looks like the produced assembler in function process_info() is wrong, there is a movslq instruction leading to results not expected on C source code level... {code} # gdb /usr/sbin/httpd.worker ./dump_new/dump/core.httpd.worker.pid8422.usr48.sig11.tim1493787964 ... (gdb) f 0 #0 process_info (r=0x7fc2840386a8, errtype=<value optimized out>) at mod_manager.c:1150 1150 ap_rprintf(r, ",Elected: %d,Read: %d,Transfered: %d,Connected: %d,Load: %d\n", (gdb) list 1145 #if AP_MODULE_MAGIC_AT_LEAST(20101223,1) 1146 proxystat = (proxy_worker_shared *) pptr; 1147 #else 1148 proxystat = (proxy_worker_stat *) pptr; 1149 #endif 1150 ap_rprintf(r, ",Elected: %d,Read: %d,Transfered: %d,Connected: %d,Load: %d\n", 1151 (int) proxystat->elected, (int) proxystat->read, (int) proxystat->transferred, 1152 (int) proxystat->busy, proxystat->lbfactor); 1153 1154 } (gdb) i registers rax 0x7fc26b59df58 140473001434968 <<<--- there should be 0x7fc2bf507f98 instead rbx 0x7fc24c056b5c 140472475806556 rcx 0x7fc2840229a8 140473415117224 rdx 0xffffffffac096100 -1408671488 rsi 0x7fc2b9d85c00 140474318347264 rdi 0x7fc2840386a8 140473415206568 rbp 0x17 0x17 rsp 0x7fc2a51f3b40 0x7fc2a51f3b40 r8 0x3036203a6c74542c 3473999597862671404 r9 0x6873756c462c6666 7526488510160463462 r10 0x3031203a74696177 3472592223112618359 r11 0x0 0 r12 0x7fc2a51f3b98 140473970670488 r13 0x7fc2840386a8 140473415206568 r14 0x17 23 r15 0x431bde82d7b634db 4835703278458516699 rip 0x7fc2b9d7d90d 0x7fc2b9d7d90d <process_info+477> eflags 0x203 [ CF IF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 (gdb) disassemble /m $rip ... 1143 pptr = (char *) ou; 0x00007fc2b9d7d8f4 <+452>: mov 0x58(%rsp),%rax 1144 pptr = pptr + ou->offset; 1145 #if AP_MODULE_MAGIC_AT_LEAST(20101223,1) 1146 proxystat = (proxy_worker_shared *) pptr; 1147 #else 1148 proxystat = (proxy_worker_stat *) pptr; 0x00007fc2b9d7d903 <+467>: movslq 0x138(%rax),%rdx <<<--- this brings wrong result into rdx 0x00007fc2b9d7d90a <+474>: add %rdx,%rax <<<--- which is then added to rax, rax should normally hold address of ou + ou->offset 1149 #endif 1150 ap_rprintf(r, ",Elected: %d,Read: %d,Transfered: %d,Connected: %d,Load: %d\n", 0x00007fc2b9d7d8f9 <+457>: lea 0x8300(%rip),%rsi # 0x7fc2b9d85c00 <<<--- the string ",Elected: %d,Read: %d,Transfered: %d,Connected: %d,Load: %d\n" 0x00007fc2b9d7d900 <+464>: mov %r13,%rdi <<<--- r parameter => 0x00007fc2b9d7d90d <+477>: mov 0xc0(%rax),%r9 <<<--- ap_rprintf params starting here proxystat->elected, ... 0x00007fc2b9d7d914 <+484>: mov 0x20(%rax),%r8 0x00007fc2b9d7d918 <+488>: mov 0x28(%rax),%rcx 0x00007fc2b9d7d91c <+492>: mov 0x30(%rax),%rdx 0x00007fc2b9d7d920 <+496>: mov 0x18(%rax),%eax 0x00007fc2b9d7d923 <+499>: mov %eax,(%rsp) 0x00007fc2b9d7d926 <+502>: xor %eax,%eax 0x00007fc2b9d7d928 <+504>: callq 0x7fc2b9d7c0f0 <ap_rprintf@plt> ... (gdb) {code} from gdb it looks like proxystat contains correct address, but it only seems so as very likely due to debugging info knowing where to look: {code} (gdb) p proxystat $2 = (proxy_worker_stat *) 0x7fc2bf507f98 (gdb) p *proxystat $11 = { status = 0, error_time = 0, retries = 0, lbstatus = 0, lbfactor = 0, transferred = 0, read = 0, elected = 0, route = '\000' <repeats 63 times>, redirect = '\000' <repeats 63 times>, context = 0x0, busy = 206158430208, lbset = 1818454389 } (gdb) {code} but actually computed value for proxystat by assembler is wrongly 0x7fc26b59df58 instead of 0x7fc2bf507f98: {code} (gdb) x/x $rax 0x7fc26b59df58: Cannot access memory at address 0x7fc26b59df58 (gdb) {code} address of ou and offset value: {code} (gdb) p ou $5 = (nodeinfo_t *) 0x7fc2bf507e58 (gdb) p /x ou->offset $7 = 0x140 (gdb) {code} their correct sum: {code} (gdb) p /x 0x7fc2bf507e58 + 0x140 $9 = 0x7fc2bf507f98 (gdb) {code} which is the same as already mentioned proxystat using debuginfo: {code} (gdb) p proxystat $2 = (proxy_worker_stat *) 0x7fc2bf507f98 (gdb) {code} so the below 2 lines in process_info() lead to computing it wrongly and they will have to be rewritten: {code} 1097 static char * process_info(request_rec *r, int *errtype) 1098 { ... 1141 pptr = (char *) ou; 1142 pptr = pptr + ou->offset; {code} {quote}