[jboss-jira] [JBoss JIRA] (WFLY-9943) Sometimes WF process does not exit on RHEL 6 (32/64 bit) and IBM JDK 8
Miroslav Novak (JIRA)
issues at jboss.org
Fri Mar 2 05:25:00 EST 2018
[ https://issues.jboss.org/browse/WFLY-9943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Miroslav Novak updated WFLY-9943:
---------------------------------
Steps to Reproduce:
*Hard to reproduce -- in less than 1% of cases in our testing environment*
*Steps to reproduce: (RHEL 6, IBM SDK 8)*
Call :shutdown CLI operation on WF12 (started with default standalone-full.xml).
Server logs that it stopped:
{code}2018-03-02 04:17:36,879 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050: JBoss EAP 7.2.0.EL12-Beta1 (WildFly Core 4.0.0.Beta1-redhat-1) stopped in 82ms{code}
but {{ps aux | grep java}} still shows that process is running:
{code}
[hudson at rhel6-x86-medium-6746 ~]$ ps aux | grep java
hudson 25506 0.2 5.6 703128 227560 ? Sl 04:17 0:09 /qa/tools/opt/ibm-java-80/bin/java -D[Standalone] -server -verbose:gc -Xloggc:/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/log/gc.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms512m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.default.multicast.address=227.0.130.48 -Djboss.messaging.cluster.password=myGreatPasswd -Djboss.messaging.group.address=227.0.130.48 -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -Djava.net.preferIPv4Stack=true -Dorg.jboss.boot.log.file=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/log/server.log -Dlogging.configuration=file:/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/configuration/logging.properties -jar /home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/jboss-modules.jar -mp /home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/modules org.jboss.as.standalone -Djboss.home.dir=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2 -Djboss.server.base.dir=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone -Djboss.bind.address=172.16.130.48 -Djboss.bind.address.unsecure=172.16.130.48 -Djboss.bind.address.management=172.16.130.48 -Djboss.bind.address.private=172.16.130.48 -c standalone-full.xml
{code}
was:
*Hard to reproduce -- in 1% of cases in our testing environment*
*Steps to reproduce: (RHEL 6, IBM SDK 8)*
Call :shutdown CLI operation on WF12 (started with default standalone-full.xml).
Server logs that it stopped:
{code}2018-03-02 04:17:36,879 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050: JBoss EAP 7.2.0.EL12-Beta1 (WildFly Core 4.0.0.Beta1-redhat-1) stopped in 82ms{code}
but {{ps aux | grep java}} still shows that process is running:
{code}
[hudson at rhel6-x86-medium-6746 ~]$ ps aux | grep java
hudson 25506 0.2 5.6 703128 227560 ? Sl 04:17 0:09 /qa/tools/opt/ibm-java-80/bin/java -D[Standalone] -server -verbose:gc -Xloggc:/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/log/gc.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms512m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.default.multicast.address=227.0.130.48 -Djboss.messaging.cluster.password=myGreatPasswd -Djboss.messaging.group.address=227.0.130.48 -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -Djava.net.preferIPv4Stack=true -Dorg.jboss.boot.log.file=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/log/server.log -Dlogging.configuration=file:/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone/configuration/logging.properties -jar /home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/jboss-modules.jar -mp /home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/modules org.jboss.as.standalone -Djboss.home.dir=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2 -Djboss.server.base.dir=/home/hudson/hudson_workspace/workspace/eap-7.x-acceptance-multinode-rhel-mnovak/4a508515/manu-eap-1.1.91/out/downloadEap1/workspace/jboss-eap-7.2/standalone -Djboss.bind.address=172.16.130.48 -Djboss.bind.address.unsecure=172.16.130.48 -Djboss.bind.address.management=172.16.130.48 -Djboss.bind.address.private=172.16.130.48 -c standalone-full.xml
{code}
> Sometimes WF process does not exit on RHEL 6 (32/64 bit) and IBM JDK 8
> ----------------------------------------------------------------------
>
> Key: WFLY-9943
> URL: https://issues.jboss.org/browse/WFLY-9943
> Project: WildFly
> Issue Type: Bug
> Components: Server
> Affects Versions: 12.0.0.Final
> Environment: RHEL 6 32/64 bit
> IBM SDK 8.0.5.10:
> {code}java version "1.8.0_161
> Java(TM) SE Runtime Environment (build 8.0.5.10 - pxi3280sr5fp10-20180214_01(SR5 FP10))
> IBM J9 VM (build 2.9, JRE 1.8.0 Linux x86-32 20180208_378436 (JIT enabled, AOT enabled)
> OpenJ9 - 39bb844
> OMR - c04ccb2
> IBM - 2321a81)
> JCL - 20180209_01 based on Oracle jdk8u161-b12
> {code}
> Reporter: Miroslav Novak
> Assignee: Jason Greene
>
> Sometimes happens that WF12 process does not exit when :shutdown CLI operation called and hangs indefinitely. This happens only RHEL 6 (32/64 bit) with IBM SDK8.
> Calling kill -3 does not create any javacore dump and only output could be gathered from pstack (attached). It indicates that JVM shutdown thread is hanging on:
> {code}
> Thread 25 (Thread 0xb7792b70 (LWP 25507)):
> #0 0x00bd4424 in __kernel_vsyscall ()
> #1 0x00b2243c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
> #2 0x004f8b1f in monitor_wait_original () from /qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9thr29.so
> #3 0x004f9957 in omrthread_monitor_wait () from /qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9thr29.so
> #4 0x0033e217 in protectedDestroyJavaVM () from /qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9vm29.so
> #5 0x0042cf6e in omrsig_protect () from /qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9prt29.so
> #6 0x0033db93 in DestroyJavaVM () from /qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libj9vm29.so
> #7 0x00815999 in DestroyJavaVM () from /qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/lib/i386/default/libjvm.so
> #8 0x00528649 in JavaMain () from /qa/tools/opt/ibm-java-i386-sdk-8.0-5.10/jre/bin/../lib/i386/jli/libjli.so
> #9 0x00b1ebc9 in start_thread () from /lib/libpthread.so.0
> #10 0x001fc04e in clone () from /lib/libc.so.6
> {code}
> There are other 24 threads which be blocking it. This appears to be problem in IBM SDK 8 in combination RHEL 6 GLIBC_2.3.2 library.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
More information about the jboss-jira
mailing list