[JBoss JIRA] (WFLY-13444) Observing High CPU in EPollArrayWrapper.epollCtl default I/O thread
by Srinivas ev (Jira)
[ https://issues.redhat.com/browse/WFLY-13444?page=com.atlassian.jira.plugi... ]
Srinivas ev updated WFLY-13444:
-------------------------------
Attachment: ThreadDump1set.zip
ThreadDump2set.zip
> Observing High CPU in EPollArrayWrapper.epollCtl default I/O thread
> -------------------------------------------------------------------
>
> Key: WFLY-13444
> URL: https://issues.redhat.com/browse/WFLY-13444
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, IO
> Affects Versions: 10.1.0.Final
> Reporter: Srinivas ev
> Assignee: Paul Ferraro
> Priority: Blocker
> Attachments: IO thread memory.PNG, IO thread.PNG, Thread consuming high CPU.PNG, ThreadDump1set.zip, ThreadDump2set.zip, stack trace in thread dump.PNG, top command output.PNG
>
>
> Facing high CPU by one of the default I/O thread In Wildfly 10.1.0 Final. Looks none of our application component thread is consuming.
> Can somebody check what is triggering this issue? I attached the thread stack to this Jira. This is blocking our releases. When the CPU cores are high in number, multiple default I/O threads will kick in consuming the CPU in 90~100% range continuously.
> Wildfly 10.1.0 Final
> Java -
> openjdk version "1.8.0_232"
> OpenJDK Runtime Environment (build 1.8.0_232-b09)
> OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode
> ---------------------------------------------------------------------------------------------------
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.fillInStackTrace(Throwable.java:784)
> at java.lang.Throwable.<init>(Throwable.java:251)
> at java.lang.Exception.<init>(Exception.java:54)
> at java.io.IOException.<init>(IOException.java:47)
> at java.nio.channels.ClosedChannelException.<init>(ClosedChannelException.java:52)
> at org.xnio.ssl.JsseStreamConduit.write(JsseStreamConduit.java:1022)
> at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150)
> at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:385)
> at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:372)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at org.xnio.conduits.WriteReadyHandler$ChannelListenerHandler.writeReady(WriteReadyHandler.java:65)
> at org.xnio.ssl.JsseStreamConduit.run(JsseStreamConduit.java:393)
> at org.xnio.ssl.JsseStreamConduit.readReady(JsseStreamConduit.java:547)
> at org.xnio.ssl.JsseStreamConduit$2.readReady(JsseStreamConduit.java:319)
> at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:567)
> -------------------------------------------------------------------------------------------
> at sun.nio.ch.EPollArrayWrapper.epollCtl(Native Method)
> at sun.nio.ch.EPollArrayWrapper.updateRegistrations(EPollArrayWrapper.java:299)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:268)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:515)
> -----------------------------------------------------------------------------------
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (WFLY-13444) Observing High CPU in EPollArrayWrapper.epollCtl default I/O thread
by Srinivas ev (Jira)
[ https://issues.redhat.com/browse/WFLY-13444?page=com.atlassian.jira.plugi... ]
Srinivas ev updated WFLY-13444:
-------------------------------
Description:
Facing high CPU by one of the default I/O thread In Wildfly 10.1.0 Final. Looks none of our application component thread is consuming.
Can somebody check what is triggering this issue? I attached the thread stack to this Jira. This is blocking our releases. When the CPU cores are high in number, multiple default I/O threads will kick in consuming the CPU in 90~100% range continuously.
Wildfly 10.1.0 Final
Java -
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode
---------------------------------------------------------------------------------------------------
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.fillInStackTrace(Throwable.java:784)
at java.lang.Throwable.<init>(Throwable.java:251)
at java.lang.Exception.<init>(Exception.java:54)
at java.io.IOException.<init>(IOException.java:47)
at java.nio.channels.ClosedChannelException.<init>(ClosedChannelException.java:52)
at org.xnio.ssl.JsseStreamConduit.write(JsseStreamConduit.java:1022)
at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150)
at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:385)
at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:372)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.WriteReadyHandler$ChannelListenerHandler.writeReady(WriteReadyHandler.java:65)
at org.xnio.ssl.JsseStreamConduit.run(JsseStreamConduit.java:393)
at org.xnio.ssl.JsseStreamConduit.readReady(JsseStreamConduit.java:547)
at org.xnio.ssl.JsseStreamConduit$2.readReady(JsseStreamConduit.java:319)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:567)
-------------------------------------------------------------------------------------------
at sun.nio.ch.EPollArrayWrapper.epollCtl(Native Method)
at sun.nio.ch.EPollArrayWrapper.updateRegistrations(EPollArrayWrapper.java:299)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:268)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:515)
-----------------------------------------------------------------------------------
was:
Facing high CPU by one of the default I/O thread In Wildfly 10.1.0 Final. Looks none of our application component thread is consuming.
Can somebody check what is triggering this issue? I attached the thread stack to this jira.
Wildfly 10.1.0 Final
Java -
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode
---------------------------------------------------------------------------------------------------
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.fillInStackTrace(Throwable.java:784)
at java.lang.Throwable.<init>(Throwable.java:251)
at java.lang.Exception.<init>(Exception.java:54)
at java.io.IOException.<init>(IOException.java:47)
at java.nio.channels.ClosedChannelException.<init>(ClosedChannelException.java:52)
at org.xnio.ssl.JsseStreamConduit.write(JsseStreamConduit.java:1022)
at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150)
at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:385)
at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:372)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.WriteReadyHandler$ChannelListenerHandler.writeReady(WriteReadyHandler.java:65)
at org.xnio.ssl.JsseStreamConduit.run(JsseStreamConduit.java:393)
at org.xnio.ssl.JsseStreamConduit.readReady(JsseStreamConduit.java:547)
at org.xnio.ssl.JsseStreamConduit$2.readReady(JsseStreamConduit.java:319)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:567)
-------------------------------------------------------------------------------------------
at sun.nio.ch.EPollArrayWrapper.epollCtl(Native Method)
at sun.nio.ch.EPollArrayWrapper.updateRegistrations(EPollArrayWrapper.java:299)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:268)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:515)
-----------------------------------------------------------------------------------
> Observing High CPU in EPollArrayWrapper.epollCtl default I/O thread
> -------------------------------------------------------------------
>
> Key: WFLY-13444
> URL: https://issues.redhat.com/browse/WFLY-13444
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, IO
> Affects Versions: 10.1.0.Final
> Reporter: Srinivas ev
> Assignee: Paul Ferraro
> Priority: Blocker
> Attachments: IO thread memory.PNG, IO thread.PNG, Thread consuming high CPU.PNG, stack trace in thread dump.PNG, top command output.PNG
>
>
> Facing high CPU by one of the default I/O thread In Wildfly 10.1.0 Final. Looks none of our application component thread is consuming.
> Can somebody check what is triggering this issue? I attached the thread stack to this Jira. This is blocking our releases. When the CPU cores are high in number, multiple default I/O threads will kick in consuming the CPU in 90~100% range continuously.
> Wildfly 10.1.0 Final
> Java -
> openjdk version "1.8.0_232"
> OpenJDK Runtime Environment (build 1.8.0_232-b09)
> OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode
> ---------------------------------------------------------------------------------------------------
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.fillInStackTrace(Throwable.java:784)
> at java.lang.Throwable.<init>(Throwable.java:251)
> at java.lang.Exception.<init>(Exception.java:54)
> at java.io.IOException.<init>(IOException.java:47)
> at java.nio.channels.ClosedChannelException.<init>(ClosedChannelException.java:52)
> at org.xnio.ssl.JsseStreamConduit.write(JsseStreamConduit.java:1022)
> at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150)
> at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:385)
> at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:372)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at org.xnio.conduits.WriteReadyHandler$ChannelListenerHandler.writeReady(WriteReadyHandler.java:65)
> at org.xnio.ssl.JsseStreamConduit.run(JsseStreamConduit.java:393)
> at org.xnio.ssl.JsseStreamConduit.readReady(JsseStreamConduit.java:547)
> at org.xnio.ssl.JsseStreamConduit$2.readReady(JsseStreamConduit.java:319)
> at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:567)
> -------------------------------------------------------------------------------------------
> at sun.nio.ch.EPollArrayWrapper.epollCtl(Native Method)
> at sun.nio.ch.EPollArrayWrapper.updateRegistrations(EPollArrayWrapper.java:299)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:268)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:515)
> -----------------------------------------------------------------------------------
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (WFLY-13444) Observing High CPU in EPollArrayWrapper.epollCtl default I/O thread
by Srinivas ev (Jira)
[ https://issues.redhat.com/browse/WFLY-13444?page=com.atlassian.jira.plugi... ]
Srinivas ev updated WFLY-13444:
-------------------------------
Priority: Blocker (was: Major)
> Observing High CPU in EPollArrayWrapper.epollCtl default I/O thread
> -------------------------------------------------------------------
>
> Key: WFLY-13444
> URL: https://issues.redhat.com/browse/WFLY-13444
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, IO
> Affects Versions: 10.1.0.Final
> Reporter: Srinivas ev
> Assignee: Paul Ferraro
> Priority: Blocker
> Attachments: IO thread memory.PNG, IO thread.PNG, Thread consuming high CPU.PNG, stack trace in thread dump.PNG, top command output.PNG
>
>
> Facing high CPU by one of the default I/O thread In Wildfly 10.1.0 Final. Looks none of our application component thread is consuming.
> Can somebody check what is triggering this issue? I attached the thread stack to this jira.
> Wildfly 10.1.0 Final
> Java -
> openjdk version "1.8.0_232"
> OpenJDK Runtime Environment (build 1.8.0_232-b09)
> OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode
> ---------------------------------------------------------------------------------------------------
> at java.lang.Throwable.fillInStackTrace(Native Method)
> at java.lang.Throwable.fillInStackTrace(Throwable.java:784)
> at java.lang.Throwable.<init>(Throwable.java:251)
> at java.lang.Exception.<init>(Exception.java:54)
> at java.io.IOException.<init>(IOException.java:47)
> at java.nio.channels.ClosedChannelException.<init>(ClosedChannelException.java:52)
> at org.xnio.ssl.JsseStreamConduit.write(JsseStreamConduit.java:1022)
> at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150)
> at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:385)
> at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:372)
> at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
> at org.xnio.conduits.WriteReadyHandler$ChannelListenerHandler.writeReady(WriteReadyHandler.java:65)
> at org.xnio.ssl.JsseStreamConduit.run(JsseStreamConduit.java:393)
> at org.xnio.ssl.JsseStreamConduit.readReady(JsseStreamConduit.java:547)
> at org.xnio.ssl.JsseStreamConduit$2.readReady(JsseStreamConduit.java:319)
> at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:567)
> -------------------------------------------------------------------------------------------
> at sun.nio.ch.EPollArrayWrapper.epollCtl(Native Method)
> at sun.nio.ch.EPollArrayWrapper.updateRegistrations(EPollArrayWrapper.java:299)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:268)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> at org.xnio.nio.WorkerThread.run(WorkerThread.java:515)
> -----------------------------------------------------------------------------------
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (WFLY-13444) Observing High CPU in EPollArrayWrapper.epollCtl default I/O thread
by Srinivas ev (Jira)
Srinivas ev created WFLY-13444:
----------------------------------
Summary: Observing High CPU in EPollArrayWrapper.epollCtl default I/O thread
Key: WFLY-13444
URL: https://issues.redhat.com/browse/WFLY-13444
Project: WildFly
Issue Type: Bug
Components: Clustering, IO
Affects Versions: 10.1.0.Final
Reporter: Srinivas ev
Assignee: Paul Ferraro
Attachments: IO thread memory.PNG, IO thread.PNG, Thread consuming high CPU.PNG, stack trace in thread dump.PNG, top command output.PNG
Facing high CPU by one of the default I/O thread In Wildfly 10.1.0 Final. Looks none of our application component thread is consuming.
Can somebody check what is triggering this issue? I attached the thread stack to this jira.
Wildfly 10.1.0 Final
Java -
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode
---------------------------------------------------------------------------------------------------
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.fillInStackTrace(Throwable.java:784)
at java.lang.Throwable.<init>(Throwable.java:251)
at java.lang.Exception.<init>(Exception.java:54)
at java.io.IOException.<init>(IOException.java:47)
at java.nio.channels.ClosedChannelException.<init>(ClosedChannelException.java:52)
at org.xnio.ssl.JsseStreamConduit.write(JsseStreamConduit.java:1022)
at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150)
at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:385)
at org.xnio.http.HttpUpgrade$HttpUpgradeState$StringWriteListener.handleEvent(HttpUpgrade.java:372)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.WriteReadyHandler$ChannelListenerHandler.writeReady(WriteReadyHandler.java:65)
at org.xnio.ssl.JsseStreamConduit.run(JsseStreamConduit.java:393)
at org.xnio.ssl.JsseStreamConduit.readReady(JsseStreamConduit.java:547)
at org.xnio.ssl.JsseStreamConduit$2.readReady(JsseStreamConduit.java:319)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:567)
-------------------------------------------------------------------------------------------
at sun.nio.ch.EPollArrayWrapper.epollCtl(Native Method)
at sun.nio.ch.EPollArrayWrapper.updateRegistrations(EPollArrayWrapper.java:299)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:268)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:515)
-----------------------------------------------------------------------------------
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years
[JBoss JIRA] (SWSQE-1143) Solve python2 vs. python3 problem on jenkins slaves
by Filip Brychta (Jira)
Filip Brychta created SWSQE-1143:
------------------------------------
Summary: Solve python2 vs. python3 problem on jenkins slaves
Key: SWSQE-1143
URL: https://issues.redhat.com/browse/SWSQE-1143
Project: Kiali QE
Issue Type: QE Task
Reporter: Filip Brychta
Assignee: Filip Brychta
Usage of python2 as a default python version on our jenkins slaves causes lot of troubles. It would be good to use python3 everywhere.
e.g. lot of problem2 with openstack client on python2:
/usr/lib/python2.7/site-packages/pkg_resources/py2_warn.py:21: UserWarning: Setuptools will stop working on Python 2
************************************************************
You are running Setuptools on Python 2, which is no longer
supported and
>>> SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release (no sooner than 2020-04-20).
Please ensure you are installing
Setuptools using pip 9.x or later or pin to `setuptools<45`
in your environment.
If you have done those things and are still encountering
this message, please follow up at
https://bit.ly/setuptools-py2-warning.
************************************************************
sys.version_info < (3,) and warnings.warn(pre + "*" * 60 + msg + "*" * 60)
Traceback (most recent call last):
File "/usr/bin/openstack", line 5, in <module>
from openstackclient.shell import main
File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 24, in <module>
from osc_lib import shell
File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 33, in <module>
from osc_lib.cli import client_config as cloud_config
File "/usr/lib/python2.7/site-packages/osc_lib/cli/client_config.py", line 18, in <module>
from openstack.config import exceptions as sdk_exceptions
File "/usr/lib/python2.7/site-packages/openstack/__init__.py", line 16, in <module>
import openstack.config
File "/usr/lib/python2.7/site-packages/openstack/config/__init__.py", line 17, in <module>
from openstack.config.loader import OpenStackConfig # noqa
File "/usr/lib/python2.7/site-packages/openstack/config/loader.py", line 33, in <module>
from openstack.config import cloud_region
File "/usr/lib/python2.7/site-packages/openstack/config/cloud_region.py", line 44, in <module>
from openstack import proxy
File "/usr/lib/python2.7/site-packages/openstack/proxy.py", line 24, in <module>
from openstack import resource
File "/usr/lib/python2.7/site-packages/openstack/resource.py", line 49, in <module>
from openstack import utils
File "/usr/lib/python2.7/site-packages/openstack/utils.py", line 13, in <module>
import queue
ImportError: No module named queue
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
6 years