[jboss-jira] [JBoss JIRA] (WFLY-9900) EJB client rarely hangs
David Lloyd (JIRA)
issues at jboss.org
Mon Feb 26 12:24:00 EST 2018
[ https://issues.jboss.org/browse/WFLY-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538407#comment-13538407 ]
David Lloyd commented on WFLY-9900:
-----------------------------------
Let me know if the stack trace looks something like this:
{noformat}
"main" #1 prio=5 os_prio=0 tid=0xf6507800 nid=0x1d9 waiting on condition [0xf667f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xe17ba650> (a org.xnio.FutureResult$1)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at org.xnio.AbstractIoFuture.await(AbstractIoFuture.java:419)
at org.xnio.AbstractIoFuture.get(AbstractIoFuture.java:535)
at org.wildfly.naming.client.remote.RemoteNamingProvider.getPeerIdentity(RemoteNamingProvider.java:126)
at org.wildfly.naming.client.remote.RemoteNamingProvider.getPeerIdentityForNaming(RemoteNamingProvider.java:106)
at org.wildfly.naming.client.remote.RemoteNamingProvider.getPeerIdentityForNaming(RemoteNamingProvider.java:53)
at org.wildfly.naming.client.NamingProvider.getPeerIdentityForNamingUsingRetry(NamingProvider.java:105)
at org.wildfly.naming.client.remote.RemoteNamingProvider.getPeerIdentityForNamingUsingRetry(RemoteNamingProvider.java:91)
at org.wildfly.naming.client.remote.RemoteContext.lambda$lookupNative$0(RemoteContext.java:189)
at org.wildfly.naming.client.remote.RemoteContext$$Lambda$170/31050501.apply(Unknown Source)
at org.wildfly.naming.client.NamingProvider.performExceptionAction(NamingProvider.java:222)
at org.wildfly.naming.client.remote.RemoteContext.performWithRetry(RemoteContext.java:100)
at org.wildfly.naming.client.remote.RemoteContext.lookupNative(RemoteContext.java:188)
at org.wildfly.naming.client.AbstractFederatingContext.lookup(AbstractFederatingContext.java:74)
at org.wildfly.naming.client.AbstractFederatingContext.lookup(AbstractFederatingContext.java:60)
at org.wildfly.naming.client.WildFlyRootContext.lookup(WildFlyRootContext.java:144)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at org.wildfly.test.integration.elytron.sasl.AbstractSaslTestBase.sendAndReceiveMsg(AbstractSaslTestBase.java:93)
at org.wildfly.test.integration.elytron.sasl.DefaultSaslConfigTestCase.lambda$testDigestInDefault$2(DefaultSaslConfigTestCase.java:97)
at org.wildfly.test.integration.elytron.sasl.DefaultSaslConfigTestCase$$Lambda$183/9484794.run(Unknown Source)
at org.wildfly.common.context.Contextual.run(Contextual.java:73)
at org.wildfly.test.integration.elytron.sasl.DefaultSaslConfigTestCase.testDigestInDefault(DefaultSaslConfigTestCase.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.jboss.arquillian.junit.Arquillian$8$1.invoke(Arquillian.java:379)
at org.jboss.arquillian.container.test.impl.execution.LocalTestExecuter.execute(LocalTestExecuter.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:85)
at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:143)
at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:114)
at org.jboss.arquillian.core.impl.EventImpl.fire(EventImpl.java:67)
at org.jboss.arquillian.container.test.impl.execution.ClientTestExecuter.execute(ClientTestExecuter.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:85)
at org.jboss.arquillian.container.test.impl.client.ContainerEventController.createContext(ContainerEventController.java:142)
at org.jboss.arquillian.container.test.impl.client.ContainerEventController.createTestContext(ContainerEventController.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92)
at org.jboss.arquillian.test.impl.TestContextHandler.createTestContext(TestContextHandler.java:130)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92)
at org.jboss.arquillian.test.impl.TestContextHandler.createClassContext(TestContextHandler.java:92)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92)
at org.jboss.arquillian.test.impl.TestContextHandler.createSuiteContext(TestContextHandler.java:73)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92)
at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:143)
at org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.test(EventTestRunnerAdaptor.java:136)
at org.jboss.arquillian.junit.Arquillian$8.evaluate(Arquillian.java:372)
at org.jboss.arquillian.junit.Arquillian$4.evaluate(Arquillian.java:246)
at org.jboss.arquillian.junit.Arquillian.multiExecute(Arquillian.java:431)
at org.jboss.arquillian.junit.Arquillian.access$200(Arquillian.java:55)
at org.jboss.arquillian.junit.Arquillian$5.evaluate(Arquillian.java:260)
at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:324)
at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:99)
at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.on(ClientBeforeAfterLifecycleEventExecuter.java:72)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.invokeObservers(EventContextImpl.java:103)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:85)
at org.jboss.arquillian.container.test.impl.client.ContainerEventController.createContext(ContainerEventController.java:142)
at org.jboss.arquillian.container.test.impl.client.ContainerEventController.createBeforeContext(ContainerEventController.java:124)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92)
at org.jboss.arquillian.test.impl.TestContextHandler.createTestContext(TestContextHandler.java:130)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92)
at org.jboss.arquillian.test.impl.TestContextHandler.createClassContext(TestContextHandler.java:92)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92)
at org.jboss.arquillian.test.impl.TestContextHandler.createSuiteContext(TestContextHandler.java:73)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.arquillian.core.impl.ObserverImpl.invoke(ObserverImpl.java:96)
at org.jboss.arquillian.core.impl.EventContextImpl.proceed(EventContextImpl.java:92)
at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:143)
at org.jboss.arquillian.core.impl.ManagerImpl.fire(ManagerImpl.java:114)
at org.jboss.arquillian.test.impl.EventTestRunnerAdaptor.fireCustomLifecycle(EventTestRunnerAdaptor.java:159)
at org.jboss.arquillian.junit.Arquillian$7.evaluate(Arquillian.java:317)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.jboss.arquillian.junit.Arquillian$2.evaluate(Arquillian.java:205)
at org.jboss.arquillian.junit.Arquillian.multiExecute(Arquillian.java:431)
at org.jboss.arquillian.junit.Arquillian.access$200(Arquillian.java:55)
at org.jboss.arquillian.junit.Arquillian$3.evaluate(Arquillian.java:219)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.jboss.arquillian.junit.Arquillian.run(Arquillian.java:167)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:161)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
{noformat}
> EJB client rarely hangs
> -----------------------
>
> Key: WFLY-9900
> URL: https://issues.jboss.org/browse/WFLY-9900
> Project: WildFly
> Issue Type: Bug
> Components: Clustering, EJB
> Affects Versions: 12.0.0.Beta1
> Reporter: Richard Janík
> Assignee: David Lloyd
>
> Seen in our clustering tests where remote EJB clients invoke beans on 4 servers in cluster, while each of those is being killed and restarted. This is done in sequence and there's plenty of time in between the kill/start/kill-another (1 minute).
> In two ouf our tests, we saw a small number of clients hang and resist termination at the end. The scenarios are:
> [jvmkill-dist-sync|https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-failover-ejb-ejbremote-jvmkill-dist-sync/36/] (36 clients)
> [shutdown-dist-sync|https://jenkins.hosts.mwqe.eng.bos.redhat.com/hudson/job/perflab_eap-7x-failover-ejb-ejbremote-shutdown-dist-sync-3owners/32/] (1 client)
> The hangs are not related to any error or warning message on the server-side. There are ClosedChannelExceptions on the client side, but these seem to be unrelated as the numbers of clients reporting the exceptions are not the same (lower) as the numbers of hanging clients.
> No thread dump as I didn't manage to reproduce yet.
> Might be related to JBEAP-12074 or JBEAP-13169.
--
This message was sent by Atlassian JIRA
(v7.5.0#75005)
More information about the jboss-jira
mailing list