[JBoss JIRA] (WFLY-13607) "SSL read loop detected" during remote EJB call; remote call blocks forever
by Victor Langelo (Jira)
[ https://issues.redhat.com/browse/WFLY-13607?page=com.atlassian.jira.plugi... ]
Victor Langelo commented on WFLY-13607:
---------------------------------------
We're seeing this same issue with WildFly 19.1.0.Final jboss-client.jar with java client running in java 11.0.6. We started noticing this after placing the server behind a nginx reverse proxy which terminates the SSL connection and forwards the http request to the WildFly server. The Wildfly server also have multiple virtual hosts. Let me know if the client messages would be useful.
> "SSL read loop detected" during remote EJB call; remote call blocks forever
> ---------------------------------------------------------------------------
>
> Key: WFLY-13607
> URL: https://issues.redhat.com/browse/WFLY-13607
> Project: WildFly
> Issue Type: Bug
> Components: Web (Undertow)
> Affects Versions: 19.1.0.Final
> Reporter: Kyle MacLeod
> Assignee: Flavia Rainone
> Priority: Major
>
> Summary: "SSL read loop detected" during remote EJB call; remote call blocks forever
> h3. Problem Description
> We are transferring data over an EJB request. The data returned from the EJB call ranges from 1-4MB in size.
> During one of these transfers, we are hitting some sort of race/timing condition which results in a "UT005076: SSL read loop detected" ERROR log.
> After the read loop detected log, the remote call blocks forever. Something is broken with the cleanup.
> Unfortunately, this is only happening on some of our servers. It's difficult to reproduce - and I don't have a test case for it.
> Other notes:
> * The issue is seen on the client side. The client is a java standalone client. All are running in docker containers. The issue is seen both running under docker-compose and under kubernetes.
> * The issue is seen with the 19.0.1.Final wildfly-client-all jar. It is NOT seen when we revert back to 18.0.0.Final wildfly-client-all jar.* It looks to me like an issue in either SslConduit or WildflyClientInputStream. There were commits post-18.0.0.Final which hit code in this area.
> h3. Logs
> What we see in the log file is these two back-to-back errors from the same thread:
> {code:language=|borderStyle=solid|theme=RDark|linenumbers=true|collapse=false}
> 2020-06-09 15:04:30,378 ERROR [io.undertow.request] [AgentServerModelRegistrationStateChangeNotification-pool-21-thread-1] UT005076: SSL read loop detected. This should not happen, please report this to the Undertow developers. Current state SslConduit{state=4, outstandingTasks=0, wrappedData=null, dataToUnwrap=null, unwrappedData=null}
> 2020-06-09 15:04:30,552 ERROR [io.undertow.request] [AgentServerModelRegistrationStateChangeNotification-pool-21-thread-1] UT005076: SSL read loop detected. This should not happen, please report this to the Undertow developers. Current state SslConduit{state=30692, outstandingTasks=0, wrappedData=null, dataToUnwrap=null, unwrappedData=null}
> {code}
> And then that same thread gets blocked forever, stuck waiting for the lock at org.wildfly.httpclient.common.WildflyClientInputStream.read(WildflyClientInputStream.java:147)
> {code:language=|borderStyle=solid|theme=RDark|linenumbers=true|collapse=true}
> "AgentServerModelRegistrationStateChangeNotification-pool-21-thread-1" #40 prio=5 os_prio=0 tid=0x00007f5cac9f8000 nid=0x47 in Object.wait() [0x00007f5c8b1d5000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at org.wildfly.httpclient.common.WildflyClientInputStream.read(WildflyClientInputStream.java:147)
> - locked <0x00000000ed9475a8> (a java.lang.Object)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at org.jboss.marshalling.SimpleDataInput.readFully(SimpleDataInput.java:175)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadByteArray(RiverUnmarshaller.java:1622)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadArray(RiverUnmarshaller.java:1690)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:355)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:231)
> at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1864)
> at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1778)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1406)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:283)
> at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:216)
> at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
> at org.wildfly.httpclient.ejb.HttpEJBReceiver$2.getResult(HttpEJBReceiver.java:207)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:613)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:544)
> at org.jboss.ejb.protocol.remote.RemotingEJBClientInterceptor.handleInvocationResult(RemotingEJBClientInterceptor.java:57)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:615)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:544)
> at org.jboss.ejb.client.TransactionPostDiscoveryInterceptor.handleInvocationResult(TransactionPostDiscoveryInterceptor.java:148)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:615)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:544)
> at org.jboss.ejb.client.DiscoveryEJBClientInterceptor.handleInvocationResult(DiscoveryEJBClientInterceptor.java:137)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:615)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:544)
> at org.jboss.ejb.client.NamingEJBClientInterceptor.handleInvocationResult(NamingEJBClientInterceptor.java:87)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:615)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:544)
> at org.jboss.ejb.client.TransactionInterceptor.handleInvocationResult(TransactionInterceptor.java:212)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:615)
> at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:544)
> at org.jboss.ejb.client.EJBClientInvocationContext.awaitResponse(EJBClientInvocationContext.java:986)
> at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:191)
> at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:125)
> at com.sun.proxy.$Proxy24.getAdapterArchive(Unknown Source)
> at com.nakina.agent.application.adaptermanager.internal.dataservice.AgentRemoteAdapterArchiveRepositoryImpl$FakeRemoteIterator.next(AgentRemoteAdapterArchiveRepositoryImpl.java:226)
> at com.nakina.agent.application.adaptermanager.internal.dataservice.AgentRemoteAdapterArchiveRepositoryImpl$FakeRemoteIterator.next(AgentRemoteAdapterArchiveRepositoryImpl.java:189)
> at com.nakina.adaptermanager.AdapterManagerImpl.addToLocalRepository(AdapterManagerImpl.java:506)
> at com.nakina.adaptermanager.AdapterManagerImpl.synchronizeLocalRepository(AdapterManagerImpl.java:617)
> at com.nakina.adaptermanager.AdapterManagerImpl.initialize(AdapterManagerImpl.java:373)
> at com.nakina.agent.application.adaptermanager.internal.dataservice.AdapterManagerFactoryImpl.getAdapterManager(AdapterManagerFactoryImpl.java:66)
> - locked <0x00000000c07fa748> (a java.lang.Object)
> at com.nakina.agent.application.adaptermanager.internal.action.StartAdapterManagerAction.execute(StartAdapterManagerAction.java:54)
> at com.nakina.oss.server.af.app.action.DefaultActionManager.executeRequest(DefaultActionManager.java:176)
> ...
> at com.nakina.oss.server.af.app.message.DefaultMessageReceiver.execute(DefaultMessageReceiver.java:65)
> at com.nakina.oss.server.af.app.action.DefaultActionManager.executeRequest(DefaultActionManager.java:176)
> at com.nakina.oss.server.af.app.message.DefaultMessageReceiver.onMessage(DefaultMessageReceiver.java:154)
> at com.nakina.oss.server.af.app.impl.LocalMessageSenderImpl$SenderRunnable.run(LocalMessageSenderImpl.java:189)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> h3. Other information:
> Java version:
> {code:language=|borderStyle=solid|theme=RDark|linenumbers=true|collapse=false}
> $ java -version
> openjdk version "1.8.0_252"
> OpenJDK Runtime Environment (build 1.8.0_252-b09)
> OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)
> {code}
> Wildfly server version:
> {code:language=|borderStyle=solid|theme=RDark|linenumbers=true|collapse=false}
> WFLYSRV0049: WildFly Full 19.1.0.Final (WildFly Core 11.1.1.Final) starting
> {code}
> wildfly-client-all version:
> * 19.0.1.Final wildfly-client-all jar.
> * NOT seen when we revert back to 18.0.0.Final wildfly-client-all jar
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 1 month
[JBoss JIRA] (SWSQE-1143) Solve python2 vs. python3 problem on jenkins slaves
by Hayk Hovsepyan (Jira)
[ https://issues.redhat.com/browse/SWSQE-1143?page=com.atlassian.jira.plugi... ]
Hayk Hovsepyan updated SWSQE-1143:
----------------------------------
Sprint: Kiali Sprint #39, Kiali Sprint #40, Kiali Sprint #41, Kiali Sprint #42, Kiali Sprint #43 (was: Kiali Sprint #39, Kiali Sprint #40, Kiali Sprint #41, Kiali Sprint #42)
> Solve python2 vs. python3 problem on jenkins slaves
> ---------------------------------------------------
>
> Key: SWSQE-1143
> URL: https://issues.redhat.com/browse/SWSQE-1143
> Project: Kiali QE
> Issue Type: QE Task
> Reporter: Filip Brychta
> Assignee: Filip Brychta
> Priority: Major
> Labels: infrastructure
>
> Usage of python2 as a default python version on our jenkins slaves causes lot of troubles. It would be good to use python3 everywhere.
> e.g. lot of problem2 with openstack client on python2:
> /usr/lib/python2.7/site-packages/pkg_resources/py2_warn.py:21: UserWarning: Setuptools will stop working on Python 2
> ************************************************************
> You are running Setuptools on Python 2, which is no longer
> supported and
> >>> SETUPTOOLS WILL STOP WORKING <<<
> in a subsequent release (no sooner than 2020-04-20).
> Please ensure you are installing
> Setuptools using pip 9.x or later or pin to `setuptools<45`
> in your environment.
> If you have done those things and are still encountering
> this message, please follow up at
> https://bit.ly/setuptools-py2-warning.
> ************************************************************
> sys.version_info < (3,) and warnings.warn(pre + "*" * 60 + msg + "*" * 60)
> Traceback (most recent call last):
> File "/usr/bin/openstack", line 5, in <module>
> from openstackclient.shell import main
> File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 24, in <module>
> from osc_lib import shell
> File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 33, in <module>
> from osc_lib.cli import client_config as cloud_config
> File "/usr/lib/python2.7/site-packages/osc_lib/cli/client_config.py", line 18, in <module>
> from openstack.config import exceptions as sdk_exceptions
> File "/usr/lib/python2.7/site-packages/openstack/__init__.py", line 16, in <module>
> import openstack.config
> File "/usr/lib/python2.7/site-packages/openstack/config/__init__.py", line 17, in <module>
> from openstack.config.loader import OpenStackConfig # noqa
> File "/usr/lib/python2.7/site-packages/openstack/config/loader.py", line 33, in <module>
> from openstack.config import cloud_region
> File "/usr/lib/python2.7/site-packages/openstack/config/cloud_region.py", line 44, in <module>
> from openstack import proxy
> File "/usr/lib/python2.7/site-packages/openstack/proxy.py", line 24, in <module>
> from openstack import resource
> File "/usr/lib/python2.7/site-packages/openstack/resource.py", line 49, in <module>
> from openstack import utils
> File "/usr/lib/python2.7/site-packages/openstack/utils.py", line 13, in <module>
> import queue
> ImportError: No module named queue
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 1 month
[JBoss JIRA] (SWSQE-1077) Kiali Test For Istio.io
by Hayk Hovsepyan (Jira)
[ https://issues.redhat.com/browse/SWSQE-1077?page=com.atlassian.jira.plugi... ]
Hayk Hovsepyan updated SWSQE-1077:
----------------------------------
Sprint: Kiali Sprint #32, Kiali Sprint #36, Kiali Sprint #37, Kiali Sprint #38, Kiali Sprint #39, Kiali Sprint #40, Kiali Sprint #41, Kiali Sprint #42, Kiali Sprint #43 (was: Kiali Sprint #32, Kiali Sprint #36, Kiali Sprint #37, Kiali Sprint #38, Kiali Sprint #39, Kiali Sprint #40, Kiali Sprint #41, Kiali Sprint #42)
> Kiali Test For Istio.io
> -----------------------
>
> Key: SWSQE-1077
> URL: https://issues.redhat.com/browse/SWSQE-1077
> Project: Kiali QE
> Issue Type: QE Task
> Reporter: Matthew Mahoney
> Priority: Major
> Labels: infrastructure
>
> The Istio.io team is requesting a Kiali test case that can be run on each submitted PR. The purpose of this test case is to validate that Kiali is not broken upon a PR submittal.
> Meeting with [~brian.avery] [~gbaufake]:
> 1) BrianA to port existing Kiali test case that was developed in old Istio.io test framework to new istiso.io framework
> https://github.com/istio/istio/blob/master/tests/e2e/tests/kiali/kiali_te...
> 2) Kiali team then to add an api test case(s) via 'curl' to validate the Istio/Kiali integration is properly functioning.
--
This message was sent by Atlassian Jira
(v7.13.8#713008)
4 years, 1 month