[
https://issues.jboss.org/browse/WFLY-9943?page=com.atlassian.jira.plugin....
]
Miroslav Novak updated WFLY-9943:
---------------------------------
Steps to Reproduce:
*Hard to reproduce -- in less than 1% of cases in our testing environment*
*Steps to reproduce: (RHEL 6, IBM SDK 8)*
Call :shutdown CLI operation on WF12 (started with default standalone-full.xml).
Server logs that it stopped:
{code}2018-03-02 04:17:36,879 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050:
JBoss EAP 7.2.0.EL12-Beta1 (WildFly Core 4.0.0.Beta1-redhat-1) stopped in 82ms{code}
but {{ps aux | grep java}} still shows that process is running:
{code}
[hudson@rhel6-x86-medium-6746 ~]$ ps aux | grep java
hudson 25506 0.2 5.6 703128 227560 ? Sl 04:17 0:09
/qa/tools/opt/ibm-java-80/bin/java -D[Standalone] -server -verbose:gc
-Xloggc:/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/log/gc.log
-XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms512m -Xmx512m
-XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true
-Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
-Djboss.default.multicast.address=227.0.130.48
-Djboss.messaging.cluster.password=myGreatPasswd
-Djboss.messaging.group.address=227.0.130.48 -Dfile.encoding=UTF-8
-Dsun.jnu.encoding=UTF-8 -Djava.net.preferIPv4Stack=true
-Dorg.jboss.boot.log.file=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/log/server.log
-Dlogging.configuration=file:/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/configuration/logging.properties
-jar
/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/jboss-modules.jar
-mp
/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/modules
org.jboss.as.standalone
-Djboss.home.dir=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2
-Djboss.server.base.dir=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone
-Djboss.bind.address=172.16.130.48 -Djboss.bind.address.unsecure=172.16.130.48
-Djboss.bind.address.management=172.16.130.48 -Djboss.bind.address.private=172.16.130.48
-c standalone-full.xml
{code}
was:
*Hard to reproduce -- in 1% of cases in our testing environment*
*Steps to reproduce: (RHEL 6, IBM SDK 8)*
Call :shutdown CLI operation on WF12 (started with default standalone-full.xml).
Server logs that it stopped:
{code}2018-03-02 04:17:36,879 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050:
JBoss EAP 7.2.0.EL12-Beta1 (WildFly Core 4.0.0.Beta1-redhat-1) stopped in 82ms{code}
but {{ps aux | grep java}} still shows that process is running:
{code}
[hudson@rhel6-x86-medium-6746 ~]$ ps aux | grep java
hudson 25506 0.2 5.6 703128 227560 ? Sl 04:17 0:09
/qa/tools/opt/ibm-java-80/bin/java -D[Standalone] -server -verbose:gc
-Xloggc:/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/log/gc.log
-XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms512m -Xmx512m
-XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true
-Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
-Djboss.default.multicast.address=227.0.130.48
-Djboss.messaging.cluster.password=myGreatPasswd
-Djboss.messaging.group.address=227.0.130.48 -Dfile.encoding=UTF-8
-Dsun.jnu.encoding=UTF-8 -Djava.net.preferIPv4Stack=true
-Dorg.jboss.boot.log.file=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/log/server.log
-Dlogging.configuration=file:/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/configuration/logging.properties
-jar
/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/jboss-modules.jar
-mp
/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/modules
org.jboss.as.standalone
-Djboss.home.dir=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2
-Djboss.server.base.dir=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone
-Djboss.bind.address=172.16.130.48 -Djboss.bind.address.unsecure=172.16.130.48
-Djboss.bind.address.management=172.16.130.48 -Djboss.bind.address.private=172.16.130.48
-c standalone-full.xml
{code}
Sometimes WF process does not exit on RHEL 6 (32/64 bit) and IBM JDK
8
----------------------------------------------------------------------
Key: WFLY-9943
URL:
https://issues.jboss.org/browse/WFLY-9943
Project: WildFly
Issue Type: Bug
Components: Server
Affects Versions: 12.0.0.Final
Environment: RHEL 6 32/64 bit
IBM SDK 8.0.5.10:
{code}java version "1.8.0_161
Java(TM) SE Runtime Environment (build 8.0.5.10 - pxi3280sr5fp10-20180214_01(SR5 FP10))
IBM J9 VM (build 2.9, JRE 1.8.0 Linux x86-32 20180208_378436 (JIT enabled, AOT enabled)
OpenJ9 - 39bb844
OMR - c04ccb2
IBM - 2321a81)
JCL - 20180209_01 based on Oracle jdk8u161-b12
{code}
Reporter: Miroslav Novak
Assignee: Jason Greene
Sometimes happens that WF12 process does not exit when :shutdown CLI operation called and
hangs indefinitely. This happens only RHEL 6 (32/64 bit) with IBM SDK8.
Calling kill -3 does not create any javacore dump and only output could be gathered from
pstack (attached). It indicates that JVM shutdown thread is hanging on:
{code}
Thread 25 (Thread 0xb7792b70 (LWP 25507)):
#0 0x00bd4424 in __kernel_vsyscall ()
#1 0x00b2243c in pthread_cond_wait@(a)GLIBC_2.3.2 () from /lib/libpthread.so.0
#2 0x004f8b1f in monitor_wait_original () from
/qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9thr29.so
#3 0x004f9957 in omrthread_monitor_wait () from
/qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9thr29.so
#4 0x0033e217 in protectedDestroyJavaVM () from
/qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9vm29.so
#5 0x0042cf6e in omrsig_protect () from
/qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9prt29.so
#6 0x0033db93 in DestroyJavaVM () from
/qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9vm29.so
#7 0x00815999 in DestroyJavaVM () from
/qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libjvm.so
#8 0x00528649 in JavaMain () from
/qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/bin/../lib/i386/jli/libjli.so
#9 0x00b1ebc9 in start_thread () from /lib/libpthread.so.0
#10 0x001fc04e in clone () from /lib/libc.so.6
{code}
There are other 24 threads which be blocking it. This appears to be problem in IBM SDK 8
in combination RHEL 6 GLIBC_2.3.2 library.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)