[JBoss JIRA] (WFLY-1536) Hangs in mixed domain testsuite
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFLY-1536?page=com.atlassian.jira.plugin.... ]
Brian Stansberry updated WFLY-1536:
-----------------------------------
Fix Version/s: 9.0.0.CR1
(was: 8.0.1.Final)
Bumping to 9, but this is likely out-of-date as I'm no longer seeing these issues in CI.
> Hangs in mixed domain testsuite
> -------------------------------
>
> Key: WFLY-1536
> URL: https://issues.jboss.org/browse/WFLY-1536
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Domain Management
> Reporter: Brian Stansberry
> Assignee: Brian Stansberry
> Fix For: 9.0.0.CR1
>
> Attachments: testsuite-hang1-server.txt, testsuite-hang2-client.txt, testsuite-hang2-server.txt
>
>
> http://lightning.mw.lab.eng.bos.redhat.com/jenkins/job/wildfly-param-pull...
> https://dl.dropboxusercontent.com/u/712508/mixed-hang-debug.zip contains details.
> 30805.dump is the client and shows the problem is occurring in the @After cleanup of MixedDomainDeploymentTest as it removes deployments from the domain.
> "main" prio=10 tid=0xf7605c00 nid=0x7856 in Object.wait() [0xf776e000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xe50275f0> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> at java.lang.Object.wait(Object.java:503)
> at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
> - locked <0xe50275f0> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
> - locked <0xe50275f0> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> at org.jboss.as.controller.client.impl.AbstractDelegatingAsyncFuture.get(AbstractDelegatingAsyncFuture.java:100)
> at org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:127)
> at org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:71)
> at org.jboss.as.controller.client.helpers.domain.impl.DomainClientImpl.execute(DomainClientImpl.java:81)
> at org.jboss.as.test.integration.domain.mixed.MixedDomainDeploymentTest.removeDeployment(MixedDomainDeploymentTest.java:441)
> at org.jboss.as.test.integration.domain.mixed.MixedDomainDeploymentTest.cleanDeployments(MixedDomainDeploymentTest.java:343)
> at org.jboss.as.test.integration.domain.mixed.MixedDomainDeploymentTest.confirmNoDeployments(MixedDomainDeploymentTest.java:163)
> 31894.dump shows the master HC. management-handler-thread is blocking on the way out (after sending the commit or rollback) waiting for a final result response from the slave.
> "management-handler-thread - 1" prio=10 tid=0xca0efc00 nid=0x7cdd in Object.wait() [0xc7469000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xeb6d6558> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> at java.lang.Object.wait(Object.java:503)
> at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
> - locked <0xeb6d6558> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
> - locked <0xeb6d6558> (a org.jboss.as.protocol.mgmt.ActiveOperationSupport$ActiveOperationImpl)
> at org.jboss.as.controller.client.impl.AbstractDelegatingAsyncFuture.get(AbstractDelegatingAsyncFuture.java:100)
> at org.jboss.as.domain.controller.operations.coordination.DomainSlaveHandler.finalizeOp(DomainSlaveHandler.java:215)
> at org.jboss.as.domain.controller.operations.coordination.DomainSlaveHandler.access$000(DomainSlaveHandler.java:58)
> at org.jboss.as.domain.controller.operations.coordination.DomainSlaveHandler$1.handleResult(DomainSlaveHandler.java:179)
> at org.jboss.as.controller.AbstractOperationContext$Step.handleRollback(AbstractOperationContext.java:805)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:763)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:738)
> (Note that the "handleRollback" method name is a red-herring -- the method should be renamed to a "handleResult" as it now is called no matter what the result.)
> 31986.dump is the slave HC. It shows it is blocking in stage DONE after having sent operationPrepared to the master, waiting for commit or rollback.
> "domain-connection-threads - 1" prio=10 tid=0xc91a8400 nid=0x7d1a waiting on condition [0xc87fe000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xea1fd410> (a java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ProxyOperationControlProxy.operationPrepared(TransactionalProtocolOperationHandler.java:173)
> at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:337)
> at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:211)
> at org.jboss.as.domain.controller.operations.coordination.ServerOperationsResolverHandler.execute(ServerOperationsResolverHandler.java:128)
> Indication is the commit or rollback message is getting lost. There are no threads shown sending or receiving it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 1 month
[JBoss JIRA] (WFLY-3090) Administrative cancellation of management ops results in closed connections
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFLY-3090?page=com.atlassian.jira.plugin.... ]
Brian Stansberry updated WFLY-3090:
-----------------------------------
Fix Version/s: 9.0.0.CR1
(was: 8.0.1.Final)
> Administrative cancellation of management ops results in closed connections
> ---------------------------------------------------------------------------
>
> Key: WFLY-3090
> URL: https://issues.jboss.org/browse/WFLY-3090
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Domain Management
> Affects Versions: 8.0.0.Final
> Reporter: Brian Stansberry
> Assignee: Brian Stansberry
> Fix For: 9.0.0.CR1
>
>
> Administrative cancellation of management ops is implemented by the thread that is handling the cancellation interrupting the thread that is executing the op. This works, but the thread executing the op can't readily distinguish between interruption do to admin action and cancellation for other reasons such as server shutdown. The OperationContext code detects interruption but restores the interrupted state to the thread, which is valid for the shutdown case. For the administrative cancellation case, it has the negative side effect of producing InterruptedExceptions later on; e.g. when the thread attempts to send data to the remote caller.
> This is particularly an issue in domain mode where the "caller" is a controlling process (HC). The InterruptedException has the effect of terminating the connection from the callee to the controlling process.
> The operation execution code needs to be able to distinguish between administrative cancellation and other causes of interruption.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 1 month
[JBoss JIRA] (WFLY-3092) ServerUpdateResultHandler does nothing
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFLY-3092?page=com.atlassian.jira.plugin.... ]
Brian Stansberry updated WFLY-3092:
-----------------------------------
Fix Version/s: 9.0.0.CR1
(was: 8.0.1.Final)
> ServerUpdateResultHandler does nothing
> --------------------------------------
>
> Key: WFLY-3092
> URL: https://issues.jboss.org/browse/WFLY-3092
> Project: WildFly
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Domain Management
> Affects Versions: 8.0.0.Final
> Reporter: Brian Stansberry
> Assignee: Brian Stansberry
> Fix For: 9.0.0.CR1
>
>
> The domain op rollout logic's AbstractServerUpdateTask.ServerUpdateResultHandler interface is serving no purpose and should be removed. It's cruft left from earlier implementations.
> The only impl stores data in a map that never gets read.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 1 month
[JBoss JIRA] (WFLY-3197) ExternalContextFactory cuts context from deployment context/requires dependency on module
by Bartosz Baranowski (JIRA)
[ https://issues.jboss.org/browse/WFLY-3197?page=com.atlassian.jira.plugin.... ]
Bartosz Baranowski updated WFLY-3197:
-------------------------------------
Description:
WFLY-2777 changed CCL for context creation(if module is set in external context factory). This essentially cuts context from deployment classes during creation time. This can cause problems when external-context-factory.module classes (ObjectFactories, etc) require deployment classes to be visible from classloader which created context( external-context.module ) and are not found.
Steps to reproduce:
1. define isolated module IM ( with classloader IM_CL )
2. create external-context-factory with module set to IM
3. create some deployment with interface, for which proxies should be looked up via external-context ( with classloader D_CL ) - deployment has no dependency on IM, since it may contain legacy classes or unwanted dependencies
4. lookup and invoke proxy
Now what will happen is during deployment external-context will be spawned just fine. However, the runtime outcome wont be pleasing.
Case 1: application and custom context dont do CCL magic
- invocation happen in deployment CCL, hence IM_CL classes are not visible and no classes required for deserialization can be loaded, this will fail ( return javax.naming.Reference instance, since ObjectFactory from IM_CL cant be found )
Case 2: application or custom context switch CCL to IM_CL for lookup
- CCL is set to IM_CL, ObjectFactory can be found, but D_CL classes cant, hence some naming error probably
Neither of above will work, unless IM_CL and D_CL have some sort of dependency on each other, which just makes module CL configured in external-context-factory irrelevant, since D_CL will require dependency on IM_CL.
Workaround:
yes, store IM_CL in context wrapper and upon first invocation obtain CCL( hopefuly D_CL), create agregating CL and set it as CCL for each context invocation.
Possible fix:
1. pass D_CL and IM_CL as part of env to factory, to allow context/context-wrapper to do some magic
2. alter ExternalContextFactory and create agregating/delegating CL, which will just iterate over IM_CL and D_CL if present to load proper classes
was:
WFLY-2777 changed CCL for context creation(if module is set in external context factory). This essentially cuts context from deployment classes during creation time. This can cause problems when external-context-factory.module classes (ObjectFactories, etc) require deployment classes to be visible from classloader which created context( external-context.module ) and are not found.
Steps to reproduce:
1. define isolated module IM ( with classloader IM_CL )
2. create external-context-factory with module set to IM
3. create some deployment with interface, for which proxies should be looked up via external-context ( with classloader D_CL ) - deployment has no dependency on IM, since it may contain legacy classes or unwanted dependencies
4. lookup and invoke proxy
Now what will happen is during deployment external-context will be spawned just fine. However, the runtime outcome wont be pleasing.
Case 1: application and custom context dont do CCL magic
- invocation happen in deployment CCL, hence IM_CL classes are not visible and no classes required for deserialization can be loaded, this will fail ( return javax.naming.Reference instance, since ObjectFactory from IM_CL cant be found )
Case 2: application or custom context switch CCL to IM_CL for lookup
- CCL is set to IM_CL, ObjectFactory can be found, but D_CL classes cant, hence some naming error probably
Neither of above will work, unless IM_CL and D_CL have some sort of dependency on each other, which just makes module CL configured in external-context-factory irrelevant, since D_CL will require dependency on IM_CL.
Workaround:
yes, store IM_CL in context wrapper and upon first invocation obtain CCL( hopefuly D_CL), create agregating CL and set it as CCL for each context invocation.
Possible fix:
1. pass D_CL and IM_CL as part of env to factory, to allow context/context-wrapper to do some magic
2. alter ExternalContextFactory and create agregating/delegating CL, which will just iterate over IM_CL and D_CL if present to load proper classes
> ExternalContextFactory cuts context from deployment context/requires dependency on module
> -----------------------------------------------------------------------------------------
>
> Key: WFLY-3197
> URL: https://issues.jboss.org/browse/WFLY-3197
> Project: WildFly
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Components: Naming
> Affects Versions: 8.0.0.Final, 8.0.1.Final
> Reporter: Bartosz Baranowski
> Assignee: Bartosz Baranowski
> Fix For: 8.0.1.Final
>
>
> WFLY-2777 changed CCL for context creation(if module is set in external context factory). This essentially cuts context from deployment classes during creation time. This can cause problems when external-context-factory.module classes (ObjectFactories, etc) require deployment classes to be visible from classloader which created context( external-context.module ) and are not found.
> Steps to reproduce:
> 1. define isolated module IM ( with classloader IM_CL )
> 2. create external-context-factory with module set to IM
> 3. create some deployment with interface, for which proxies should be looked up via external-context ( with classloader D_CL ) - deployment has no dependency on IM, since it may contain legacy classes or unwanted dependencies
> 4. lookup and invoke proxy
> Now what will happen is during deployment external-context will be spawned just fine. However, the runtime outcome wont be pleasing.
> Case 1: application and custom context dont do CCL magic
> - invocation happen in deployment CCL, hence IM_CL classes are not visible and no classes required for deserialization can be loaded, this will fail ( return javax.naming.Reference instance, since ObjectFactory from IM_CL cant be found )
> Case 2: application or custom context switch CCL to IM_CL for lookup
> - CCL is set to IM_CL, ObjectFactory can be found, but D_CL classes cant, hence some naming error probably
> Neither of above will work, unless IM_CL and D_CL have some sort of dependency on each other, which just makes module CL configured in external-context-factory irrelevant, since D_CL will require dependency on IM_CL.
> Workaround:
> yes, store IM_CL in context wrapper and upon first invocation obtain CCL( hopefuly D_CL), create agregating CL and set it as CCL for each context invocation.
> Possible fix:
> 1. pass D_CL and IM_CL as part of env to factory, to allow context/context-wrapper to do some magic
> 2. alter ExternalContextFactory and create agregating/delegating CL, which will just iterate over IM_CL and D_CL if present to load proper classes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 1 month
[JBoss JIRA] (WFLY-3197) ExternalContextFactory cuts context from deployment context/requires dependency on module
by Bartosz Baranowski (JIRA)
[ https://issues.jboss.org/browse/WFLY-3197?page=com.atlassian.jira.plugin.... ]
Bartosz Baranowski updated WFLY-3197:
-------------------------------------
Fix Version/s: 8.0.1.Final
Affects Version/s: 8.0.0.Final
8.0.1.Final
> ExternalContextFactory cuts context from deployment context/requires dependency on module
> -----------------------------------------------------------------------------------------
>
> Key: WFLY-3197
> URL: https://issues.jboss.org/browse/WFLY-3197
> Project: WildFly
> Issue Type: Enhancement
> Security Level: Public(Everyone can see)
> Components: Naming
> Affects Versions: 8.0.0.Final, 8.0.1.Final
> Reporter: Bartosz Baranowski
> Assignee: Bartosz Baranowski
> Fix For: 8.0.1.Final
>
>
> WFLY-2777 changed CCL for context creation(if module is set in external context factory). This essentially cuts context from deployment classes during creation time. This can cause problems when external-context-factory.module classes (ObjectFactories, etc) require deployment classes to be visible from classloader which created context( external-context.module ) and are not found.
> Steps to reproduce:
> 1. define isolated module IM ( with classloader IM_CL )
> 2. create external-context-factory with module set to IM
> 3. create some deployment with interface, for which proxies should be looked up via external-context ( with classloader D_CL ) - deployment has no dependency on IM, since it may contain legacy classes or unwanted dependencies
> 4. lookup and invoke proxy
> Now what will happen is during deployment external-context will be spawned just fine. However, the runtime outcome wont be pleasing.
> Case 1: application and custom context dont do CCL magic
> - invocation happen in deployment CCL, hence IM_CL classes are not visible and no classes required for deserialization can be loaded, this will fail ( return javax.naming.Reference instance, since ObjectFactory from IM_CL cant be found )
> Case 2: application or custom context switch CCL to IM_CL for lookup
> - CCL is set to IM_CL, ObjectFactory can be found, but D_CL classes cant, hence some naming error probably
> Neither of above will work, unless IM_CL and D_CL have some sort of dependency on each other, which just makes module CL configured in external-context-factory irrelevant, since D_CL will require dependency on IM_CL.
> Workaround:
> yes, store IM_CL in context wrapper and upon first invocation obtain CCL( hopefuly D_CL), create agregating CL and set it as CCL for each context invocation.
> Possible fix:
> 1. pass D_CL and IM_CL as part of env to factory, to allow context/context-wrapper to do some magic
> 2. alter ExternalContextFactory and create agregating/delegating CL, which will just iterate over IM_CL and D_CL if present to load proper classes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
10 years, 1 month