[JBoss JIRA] (WFLY-6591) Expired/invalidated session meta-data entries not removed when using optimistic locking or READ_COMMITTED isolation
by Paul Ferraro (JIRA)
[ https://issues.jboss.org/browse/WFLY-6591?page=com.atlassian.jira.plugin.... ]
Paul Ferraro updated WFLY-6591:
-------------------------------
Summary: Expired/invalidated session meta-data entries not removed when using optimistic locking or READ_COMMITTED isolation (was: Memory leak of expired session in Infinispan cache with optimistic locking)
> Expired/invalidated session meta-data entries not removed when using optimistic locking or READ_COMMITTED isolation
> -------------------------------------------------------------------------------------------------------------------
>
> Key: WFLY-6591
> URL: https://issues.jboss.org/browse/WFLY-6591
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.Final, 10.1.0.Final
> Reporter: Gabriel Lavoie
> Assignee: Paul Ferraro
>
> When using optimistic locking on the web cache, data from expired sessions is not removed from the web cache by the org.wildfly.clustering.web.infinispan.session.SessionExpirationScheduler and leaks forever.
> Session seems to be invalidated correctly and isn't accessible anymore.
> After tracing the code, org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.remove() doesn't remove the metadata entries when "this.properties.isLockOnWrite()" returns false (which is always the case for optimistic locking). Then in org.wildfly.clustering.web.infinispan.session.InfinispanSessionFactory.remove(), the call to attributesFactory.remove(id) also gets skipped.
> We've tried to adjust the code of org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.remove() to the following:
> {code}
> private boolean remove(String id, Cache<SessionCreationMetaDataKey, SessionCreationMetaDataEntry<L>> creationMetaDataCache) {
> SessionCreationMetaDataKey key = new SessionCreationMetaDataKey(id);
> if (this.properties.isLockOnWrite()) {
> if (creationMetaDataCache.getAdvancedCache().withFlags(Flag.ZERO_LOCK_ACQUISITION_TIMEOUT, Flag.FAIL_SILENTLY).lock(key)) {
> creationMetaDataCache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).remove(key);
> this.accessMetaDataCache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).remove(new SessionAccessMetaDataKey(id));
> return true;
> }
> return false;
> } else {
> creationMetaDataCache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).remove(key);
> this.accessMetaDataCache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).remove(new SessionAccessMetaDataKey(id));
> return true;
> }
> }
> {code}
> With that change, the memory leaks disappears.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 2 months
[JBoss JIRA] (WFLY-6591) Memory leak of expired session in Infinispan cache with optimistic locking
by Paul Ferraro (JIRA)
[ https://issues.jboss.org/browse/WFLY-6591?page=com.atlassian.jira.plugin.... ]
Paul Ferraro commented on WFLY-6591:
------------------------------------
Acknowledged. Fix on the way.
Regarding ISPN-694 - session expiration support is not sufficiently complete. To date, Infinispan does not yet support expiration notifications for passivated cache entries, i.e. expiration from a cache store.
> Memory leak of expired session in Infinispan cache with optimistic locking
> --------------------------------------------------------------------------
>
> Key: WFLY-6591
> URL: https://issues.jboss.org/browse/WFLY-6591
> Project: WildFly
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 10.0.0.Final, 10.1.0.Final
> Reporter: Gabriel Lavoie
> Assignee: Paul Ferraro
>
> When using optimistic locking on the web cache, data from expired sessions is not removed from the web cache by the org.wildfly.clustering.web.infinispan.session.SessionExpirationScheduler and leaks forever.
> Session seems to be invalidated correctly and isn't accessible anymore.
> After tracing the code, org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.remove() doesn't remove the metadata entries when "this.properties.isLockOnWrite()" returns false (which is always the case for optimistic locking). Then in org.wildfly.clustering.web.infinispan.session.InfinispanSessionFactory.remove(), the call to attributesFactory.remove(id) also gets skipped.
> We've tried to adjust the code of org.wildfly.clustering.web.infinispan.session.InfinispanSessionMetaDataFactory.remove() to the following:
> {code}
> private boolean remove(String id, Cache<SessionCreationMetaDataKey, SessionCreationMetaDataEntry<L>> creationMetaDataCache) {
> SessionCreationMetaDataKey key = new SessionCreationMetaDataKey(id);
> if (this.properties.isLockOnWrite()) {
> if (creationMetaDataCache.getAdvancedCache().withFlags(Flag.ZERO_LOCK_ACQUISITION_TIMEOUT, Flag.FAIL_SILENTLY).lock(key)) {
> creationMetaDataCache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).remove(key);
> this.accessMetaDataCache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).remove(new SessionAccessMetaDataKey(id));
> return true;
> }
> return false;
> } else {
> creationMetaDataCache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).remove(key);
> this.accessMetaDataCache.getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES).remove(new SessionAccessMetaDataKey(id));
> return true;
> }
> }
> {code}
> With that change, the memory leaks disappears.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 2 months
[JBoss JIRA] (WFLY-6592) NPE thrown during application redeployment, slaves taken offline
by James Perkins (JIRA)
[ https://issues.jboss.org/browse/WFLY-6592?page=com.atlassian.jira.plugin.... ]
James Perkins commented on WFLY-6592:
-------------------------------------
Perfect thanks! It shouldn't make a difference. It looks like the deployments parent directory is {{null}} for some reason. Or possibly the parent gets set to an invalid file and {{File.list()}} returns {{null}}. That's probably more likely what's happening looking at it.
> NPE thrown during application redeployment, slaves taken offline
> ----------------------------------------------------------------
>
> Key: WFLY-6592
> URL: https://issues.jboss.org/browse/WFLY-6592
> Project: WildFly
> Issue Type: Bug
> Affects Versions: 10.0.0.Final
> Reporter: Matthew Casperson
> Assignee: Jason Greene
>
> We have some development Wildfly 10.0.0 servers running as slaves in a domain that frequently have WAR files redeployed. We have noticed that these slaves will often go offline after a redeployment of WAR files with the following stack trace:
> {code}
> 2016-05-06 05:05:51,306 ERROR [org.jboss.as.controller.management-operation] (Host Controller Service Threads - 1012) WFLYCTL0190: Step handler org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler@3f68226b for operation {"operation" => "full-replace-deployment","name" => "whatever.war","enabled" => true,"content" => [{"hash" => bytes { 0x5d, 0x12, 0x18, 0x2b, 0x1c, 0x86, 0x71, 0x27, 0x08, 0x3d, 0xf1, 0x75, 0x08, 0x29, 0xa6, 0x49, 0x1f, 0x16, 0xe8, 0x22 }}],"operation-headers" => {"access-mechanism" => "NATIVE","domain-uuid" => "802ab616-dd2c-4081-a79c-c4d54e14c384","push-to-servers" => undefined},"address" => [],"runtime-name" => undefined} at address [] failed handling operation rollback -- java.lang.NullPointerException: java.lang.NullPointerException
> at org.jboss.as.repository.LocalDeploymentFileRepository.deleteDeployment(LocalDeploymentFileRepository.java:59)
> at org.jboss.as.host.controller.RemoteDomainConnectionService$RemoteFileRepository.deleteDeployment(RemoteDomainConnectionService.java:756)
> at org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler$1.handleResult(DeploymentFullReplaceHandler.java:181)
> at org.jboss.as.controller.AbstractOperationContext$Step.invokeResultHandler(AbstractOperationContext.java:1384)
> at org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1366)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1328)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:1311)
> at org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185)
> at org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767)
> at org.jboss.as.controller.AbstractOperationContext.executeDoneStage(AbstractOperationContext.java:753)
> at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:680)
> at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
> at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1344)
> at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:392)
> at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:217)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler.internalExecute(TransactionalProtocolOperationHandler.java:247)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler.doExecute(TransactionalProtocolOperationHandler.java:185)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:138)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:134)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:81)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:157)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2.execute(TransactionalProtocolOperationHandler.java:153)
> at org.jboss.as.protocol.mgmt.AbstractMessageHandler$ManagementRequestContextImpl$1.doExecute(AbstractMessageHandler.java:363)
> at org.jboss.as.protocol.mgmt.AbstractMessageHandler$AsyncTaskRunner.run(AbstractMessageHandler.java:472)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> {code}
> This error will usually only happen for 2 out of the 4 identically configured slaves, and seems to happen randomly, although frequently enough.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 2 months
[JBoss JIRA] (WFLY-6592) NPE thrown during application redeployment, slaves taken offline
by Matthew Casperson (JIRA)
[ https://issues.jboss.org/browse/WFLY-6592?page=com.atlassian.jira.plugin.... ]
Matthew Casperson edited comment on WFLY-6592 at 5/5/16 6:26 PM:
-----------------------------------------------------------------
We use the CLI to --force overwrite the deployments on the domain controller.
I'm not sure if it makes a difference, but we are in the process of migrating from Tomcat to WildFly, so we are deploying WARs with Tomcat version numbers in the filename but resetting the name in the deployment command i.e.
deploy /usr/bamboo/xml-data/build-dir/41811987-43712537/services##20160504.0614-r66d0ca5f382826538ea28e24713b2f9abba3ec27.war --name=services.war --force
was (Author: mcasperson):
We use the CLI to --force overwrite the deployments on the domain controller.
> NPE thrown during application redeployment, slaves taken offline
> ----------------------------------------------------------------
>
> Key: WFLY-6592
> URL: https://issues.jboss.org/browse/WFLY-6592
> Project: WildFly
> Issue Type: Bug
> Affects Versions: 10.0.0.Final
> Reporter: Matthew Casperson
> Assignee: Jason Greene
>
> We have some development Wildfly 10.0.0 servers running as slaves in a domain that frequently have WAR files redeployed. We have noticed that these slaves will often go offline after a redeployment of WAR files with the following stack trace:
> {code}
> 2016-05-06 05:05:51,306 ERROR [org.jboss.as.controller.management-operation] (Host Controller Service Threads - 1012) WFLYCTL0190: Step handler org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler@3f68226b for operation {"operation" => "full-replace-deployment","name" => "whatever.war","enabled" => true,"content" => [{"hash" => bytes { 0x5d, 0x12, 0x18, 0x2b, 0x1c, 0x86, 0x71, 0x27, 0x08, 0x3d, 0xf1, 0x75, 0x08, 0x29, 0xa6, 0x49, 0x1f, 0x16, 0xe8, 0x22 }}],"operation-headers" => {"access-mechanism" => "NATIVE","domain-uuid" => "802ab616-dd2c-4081-a79c-c4d54e14c384","push-to-servers" => undefined},"address" => [],"runtime-name" => undefined} at address [] failed handling operation rollback -- java.lang.NullPointerException: java.lang.NullPointerException
> at org.jboss.as.repository.LocalDeploymentFileRepository.deleteDeployment(LocalDeploymentFileRepository.java:59)
> at org.jboss.as.host.controller.RemoteDomainConnectionService$RemoteFileRepository.deleteDeployment(RemoteDomainConnectionService.java:756)
> at org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler$1.handleResult(DeploymentFullReplaceHandler.java:181)
> at org.jboss.as.controller.AbstractOperationContext$Step.invokeResultHandler(AbstractOperationContext.java:1384)
> at org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1366)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1328)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:1311)
> at org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185)
> at org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767)
> at org.jboss.as.controller.AbstractOperationContext.executeDoneStage(AbstractOperationContext.java:753)
> at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:680)
> at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
> at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1344)
> at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:392)
> at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:217)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler.internalExecute(TransactionalProtocolOperationHandler.java:247)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler.doExecute(TransactionalProtocolOperationHandler.java:185)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:138)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:134)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:81)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:157)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2.execute(TransactionalProtocolOperationHandler.java:153)
> at org.jboss.as.protocol.mgmt.AbstractMessageHandler$ManagementRequestContextImpl$1.doExecute(AbstractMessageHandler.java:363)
> at org.jboss.as.protocol.mgmt.AbstractMessageHandler$AsyncTaskRunner.run(AbstractMessageHandler.java:472)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> {code}
> This error will usually only happen for 2 out of the 4 identically configured slaves, and seems to happen randomly, although frequently enough.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 2 months
[JBoss JIRA] (WFLY-6592) NPE thrown during application redeployment, slaves taken offline
by Matthew Casperson (JIRA)
[ https://issues.jboss.org/browse/WFLY-6592?page=com.atlassian.jira.plugin.... ]
Matthew Casperson commented on WFLY-6592:
-----------------------------------------
We use the CLI to --force overwrite the deployments on the domain controller.
> NPE thrown during application redeployment, slaves taken offline
> ----------------------------------------------------------------
>
> Key: WFLY-6592
> URL: https://issues.jboss.org/browse/WFLY-6592
> Project: WildFly
> Issue Type: Bug
> Affects Versions: 10.0.0.Final
> Reporter: Matthew Casperson
> Assignee: Jason Greene
>
> We have some development Wildfly 10.0.0 servers running as slaves in a domain that frequently have WAR files redeployed. We have noticed that these slaves will often go offline after a redeployment of WAR files with the following stack trace:
> {code}
> 2016-05-06 05:05:51,306 ERROR [org.jboss.as.controller.management-operation] (Host Controller Service Threads - 1012) WFLYCTL0190: Step handler org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler@3f68226b for operation {"operation" => "full-replace-deployment","name" => "whatever.war","enabled" => true,"content" => [{"hash" => bytes { 0x5d, 0x12, 0x18, 0x2b, 0x1c, 0x86, 0x71, 0x27, 0x08, 0x3d, 0xf1, 0x75, 0x08, 0x29, 0xa6, 0x49, 0x1f, 0x16, 0xe8, 0x22 }}],"operation-headers" => {"access-mechanism" => "NATIVE","domain-uuid" => "802ab616-dd2c-4081-a79c-c4d54e14c384","push-to-servers" => undefined},"address" => [],"runtime-name" => undefined} at address [] failed handling operation rollback -- java.lang.NullPointerException: java.lang.NullPointerException
> at org.jboss.as.repository.LocalDeploymentFileRepository.deleteDeployment(LocalDeploymentFileRepository.java:59)
> at org.jboss.as.host.controller.RemoteDomainConnectionService$RemoteFileRepository.deleteDeployment(RemoteDomainConnectionService.java:756)
> at org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler$1.handleResult(DeploymentFullReplaceHandler.java:181)
> at org.jboss.as.controller.AbstractOperationContext$Step.invokeResultHandler(AbstractOperationContext.java:1384)
> at org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1366)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1328)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:1311)
> at org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185)
> at org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767)
> at org.jboss.as.controller.AbstractOperationContext.executeDoneStage(AbstractOperationContext.java:753)
> at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:680)
> at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
> at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1344)
> at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:392)
> at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:217)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler.internalExecute(TransactionalProtocolOperationHandler.java:247)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler.doExecute(TransactionalProtocolOperationHandler.java:185)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:138)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:134)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:81)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:157)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2.execute(TransactionalProtocolOperationHandler.java:153)
> at org.jboss.as.protocol.mgmt.AbstractMessageHandler$ManagementRequestContextImpl$1.doExecute(AbstractMessageHandler.java:363)
> at org.jboss.as.protocol.mgmt.AbstractMessageHandler$AsyncTaskRunner.run(AbstractMessageHandler.java:472)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> {code}
> This error will usually only happen for 2 out of the 4 identically configured slaves, and seems to happen randomly, although frequently enough.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 2 months
[JBoss JIRA] (WFLY-6592) NPE thrown during application redeployment, slaves taken offline
by James Perkins (JIRA)
[ https://issues.jboss.org/browse/WFLY-6592?page=com.atlassian.jira.plugin.... ]
James Perkins commented on WFLY-6592:
-------------------------------------
How are they deployed/redeployed?
> NPE thrown during application redeployment, slaves taken offline
> ----------------------------------------------------------------
>
> Key: WFLY-6592
> URL: https://issues.jboss.org/browse/WFLY-6592
> Project: WildFly
> Issue Type: Bug
> Affects Versions: 10.0.0.Final
> Reporter: Matthew Casperson
> Assignee: Jason Greene
>
> We have some development Wildfly 10.0.0 servers running as slaves in a domain that frequently have WAR files redeployed. We have noticed that these slaves will often go offline after a redeployment of WAR files with the following stack trace:
> {code}
> 2016-05-06 05:05:51,306 ERROR [org.jboss.as.controller.management-operation] (Host Controller Service Threads - 1012) WFLYCTL0190: Step handler org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler@3f68226b for operation {"operation" => "full-replace-deployment","name" => "whatever.war","enabled" => true,"content" => [{"hash" => bytes { 0x5d, 0x12, 0x18, 0x2b, 0x1c, 0x86, 0x71, 0x27, 0x08, 0x3d, 0xf1, 0x75, 0x08, 0x29, 0xa6, 0x49, 0x1f, 0x16, 0xe8, 0x22 }}],"operation-headers" => {"access-mechanism" => "NATIVE","domain-uuid" => "802ab616-dd2c-4081-a79c-c4d54e14c384","push-to-servers" => undefined},"address" => [],"runtime-name" => undefined} at address [] failed handling operation rollback -- java.lang.NullPointerException: java.lang.NullPointerException
> at org.jboss.as.repository.LocalDeploymentFileRepository.deleteDeployment(LocalDeploymentFileRepository.java:59)
> at org.jboss.as.host.controller.RemoteDomainConnectionService$RemoteFileRepository.deleteDeployment(RemoteDomainConnectionService.java:756)
> at org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler$1.handleResult(DeploymentFullReplaceHandler.java:181)
> at org.jboss.as.controller.AbstractOperationContext$Step.invokeResultHandler(AbstractOperationContext.java:1384)
> at org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1366)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1328)
> at org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:1311)
> at org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185)
> at org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767)
> at org.jboss.as.controller.AbstractOperationContext.executeDoneStage(AbstractOperationContext.java:753)
> at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:680)
> at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
> at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1344)
> at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:392)
> at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:217)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler.internalExecute(TransactionalProtocolOperationHandler.java:247)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler.doExecute(TransactionalProtocolOperationHandler.java:185)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:138)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:134)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:81)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:157)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2.execute(TransactionalProtocolOperationHandler.java:153)
> at org.jboss.as.protocol.mgmt.AbstractMessageHandler$ManagementRequestContextImpl$1.doExecute(AbstractMessageHandler.java:363)
> at org.jboss.as.protocol.mgmt.AbstractMessageHandler$AsyncTaskRunner.run(AbstractMessageHandler.java:472)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> {code}
> This error will usually only happen for 2 out of the 4 identically configured slaves, and seems to happen randomly, although frequently enough.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 2 months
[JBoss JIRA] (WFLY-6592) NPE thrown during application redeployment, slaves taken offline
by Matthew Casperson (JIRA)
Matthew Casperson created WFLY-6592:
---------------------------------------
Summary: NPE thrown during application redeployment, slaves taken offline
Key: WFLY-6592
URL: https://issues.jboss.org/browse/WFLY-6592
Project: WildFly
Issue Type: Bug
Affects Versions: 10.0.0.Final
Reporter: Matthew Casperson
Assignee: Jason Greene
We have some development Wildfly 10.0.0 servers running as slaves in a domain that frequently have WAR files redeployed. We have noticed that these slaves will often go offline after a redeployment of WAR files with the following stack trace:
{code}
2016-05-06 05:05:51,306 ERROR [org.jboss.as.controller.management-operation] (Host Controller Service Threads - 1012) WFLYCTL0190: Step handler org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler@3f68226b for operation {"operation" => "full-replace-deployment","name" => "whatever.war","enabled" => true,"content" => [{"hash" => bytes { 0x5d, 0x12, 0x18, 0x2b, 0x1c, 0x86, 0x71, 0x27, 0x08, 0x3d, 0xf1, 0x75, 0x08, 0x29, 0xa6, 0x49, 0x1f, 0x16, 0xe8, 0x22 }}],"operation-headers" => {"access-mechanism" => "NATIVE","domain-uuid" => "802ab616-dd2c-4081-a79c-c4d54e14c384","push-to-servers" => undefined},"address" => [],"runtime-name" => undefined} at address [] failed handling operation rollback -- java.lang.NullPointerException: java.lang.NullPointerException
at org.jboss.as.repository.LocalDeploymentFileRepository.deleteDeployment(LocalDeploymentFileRepository.java:59)
at org.jboss.as.host.controller.RemoteDomainConnectionService$RemoteFileRepository.deleteDeployment(RemoteDomainConnectionService.java:756)
at org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler$1.handleResult(DeploymentFullReplaceHandler.java:181)
at org.jboss.as.controller.AbstractOperationContext$Step.invokeResultHandler(AbstractOperationContext.java:1384)
at org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1366)
at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1328)
at org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:1311)
at org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185)
at org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767)
at org.jboss.as.controller.AbstractOperationContext.executeDoneStage(AbstractOperationContext.java:753)
at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:680)
at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1344)
at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:392)
at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:217)
at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler.internalExecute(TransactionalProtocolOperationHandler.java:247)
at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler.doExecute(TransactionalProtocolOperationHandler.java:185)
at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:138)
at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:134)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:81)
at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:157)
at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:153)
at java.security.AccessController.doPrivileged(Native Method)
at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2.execute(TransactionalProtocolOperationHandler.java:153)
at org.jboss.as.protocol.mgmt.AbstractMessageHandler$ManagementRequestContextImpl$1.doExecute(AbstractMessageHandler.java:363)
at org.jboss.as.protocol.mgmt.AbstractMessageHandler$AsyncTaskRunner.run(AbstractMessageHandler.java:472)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
{code}
This error will usually only happen for 2 out of the 4 identically configured slaves, and seems to happen randomly, although frequently enough.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 2 months
[JBoss JIRA] (WFCORE-1531) The HC shutdown operation should accept a restart-servers param like reload does
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-1531?page=com.atlassian.jira.plugi... ]
Brian Stansberry commented on WFCORE-1531:
------------------------------------------
Note there needs to be some validation, as restart=false and restart-servers=false is an invalid combination. The user isn't allowed to ask for servers without an HC.
The default for restart-servers would be true, which is the current behavior.
> The HC shutdown operation should accept a restart-servers param like reload does
> --------------------------------------------------------------------------------
>
> Key: WFCORE-1531
> URL: https://issues.jboss.org/browse/WFCORE-1531
> Project: WildFly Core
> Issue Type: Feature Request
> Components: CLI, Domain Management
> Reporter: Brian Stansberry
>
> If a user uses a management op to restart the HC, e.g. one of these:
> {code}
> [domain@localhost:9990 /] /host=master:shutdown(restart=true)
> {"outcome" => "success"}
> [domain@localhost:9990 /] shutdown --host=master --restart=true
> {code}
> then they should also be able to configure restart-servers=false so the servers remain alive and reconnect to the HC when the PC spawns a new one.
> Doing "kill -9 <hc_pid>" works fine, with the PC spawning a new HC and the servers reconnecting, so it seems the main thing here is adding the parameter and wiring in the mechanicsm such that the ServerInventory doesn't stop the servers. Perhaps the same thing can be done as HostProcessReloadHandler does, using HostRunningModeControl.setRestartMode().
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 2 months
[JBoss JIRA] (WFCORE-1531) The HC shutdown operation should accept a restart-servers param like reload does
by Brian Stansberry (JIRA)
Brian Stansberry created WFCORE-1531:
----------------------------------------
Summary: The HC shutdown operation should accept a restart-servers param like reload does
Key: WFCORE-1531
URL: https://issues.jboss.org/browse/WFCORE-1531
Project: WildFly Core
Issue Type: Feature Request
Components: CLI, Domain Management
Reporter: Brian Stansberry
If a user uses a management op to restart the HC, e.g. one of these:
{code}
[domain@localhost:9990 /] /host=master:shutdown(restart=true)
{"outcome" => "success"}
[domain@localhost:9990 /] shutdown --host=master --restart=true
{code}
then they should also be able to configure restart-servers=false so the servers remain alive and reconnect to the HC when the PC spawns a new one.
Doing "kill -9 <hc_pid>" works fine, with the PC spawning a new HC and the servers reconnecting, so it seems the main thing here is adding the parameter and wiring in the mechanicsm such that the ServerInventory doesn't stop the servers. Perhaps the same thing can be done as HostProcessReloadHandler does, using HostRunningModeControl.setRestartMode().
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 2 months