[jboss-jira] [JBoss JIRA] (WFLY-6592) NPE thrown during application redeployment, slaves taken offline

[Matthew Casperson (JIRA)]

    [ https://issues.jboss.org/browse/WFLY-6592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201812#comment-13201812 ] 

Matthew Casperson commented on WFLY-6592:
-----------------------------------------

Thinking about this error, I suspect this is what happens to cause it:

1. We deploy an app to a domain controller with 2 slaves
2. The deployment on one slave fails because of memory issues (not a wildfly bug, more of a regular "os doesn't have swap memory and we assigned to much to java" issue)
3. At this point the deployment of the app is rolled back, but the server that didn't run out of memory doesn't have anything to roll back, causing the NPE.

I haven't tested this theory yet, but it could be more than a coincidence. 

> NPE thrown during application redeployment, slaves taken offline
> ----------------------------------------------------------------
>
>                 Key: WFLY-6592
>                 URL: https://issues.jboss.org/browse/WFLY-6592
>             Project: WildFly
>          Issue Type: Bug
>    Affects Versions: 10.0.0.Final
>            Reporter: Matthew Casperson
>            Assignee: Jason Greene
>
> We have some development Wildfly 10.0.0 servers running as slaves in a domain that frequently have WAR files redeployed. We have noticed that these slaves will often go offline after a redeployment of WAR files with the following stack trace:
> {code}
> 2016-05-06 05:05:51,306 ERROR [org.jboss.as.controller.management-operation] (Host Controller Service Threads - 1012) WFLYCTL0190: Step handler org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler at 3f68226b for operation {"operation" => "full-replace-deployment","name" => "whatever.war","enabled" => true,"content" => [{"hash" => bytes { 0x5d, 0x12, 0x18, 0x2b, 0x1c, 0x86, 0x71, 0x27, 0x08, 0x3d, 0xf1, 0x75, 0x08, 0x29, 0xa6, 0x49, 0x1f, 0x16, 0xe8, 0x22 }}],"operation-headers" => {"access-mechanism" => "NATIVE","domain-uuid" => "802ab616-dd2c-4081-a79c-c4d54e14c384","push-to-servers" => undefined},"address" => [],"runtime-name" => undefined} at address [] failed handling operation rollback -- java.lang.NullPointerException: java.lang.NullPointerException
>         at org.jboss.as.repository.LocalDeploymentFileRepository.deleteDeployment(LocalDeploymentFileRepository.java:59)
>         at org.jboss.as.host.controller.RemoteDomainConnectionService$RemoteFileRepository.deleteDeployment(RemoteDomainConnectionService.java:756)
>         at org.jboss.as.domain.controller.operations.deployment.DeploymentFullReplaceHandler$1.handleResult(DeploymentFullReplaceHandler.java:181)
>         at org.jboss.as.controller.AbstractOperationContext$Step.invokeResultHandler(AbstractOperationContext.java:1384)
>         at org.jboss.as.controller.AbstractOperationContext$Step.handleResult(AbstractOperationContext.java:1366)
>         at org.jboss.as.controller.AbstractOperationContext$Step.finalizeInternal(AbstractOperationContext.java:1328)
>         at org.jboss.as.controller.AbstractOperationContext$Step.finalizeStep(AbstractOperationContext.java:1311)
>         at org.jboss.as.controller.AbstractOperationContext$Step.access$300(AbstractOperationContext.java:1185)
>         at org.jboss.as.controller.AbstractOperationContext.executeResultHandlerPhase(AbstractOperationContext.java:767)
>         at org.jboss.as.controller.AbstractOperationContext.executeDoneStage(AbstractOperationContext.java:753)
>         at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:680)
>         at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
>         at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1344)
>         at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:392)
>         at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:217)
>         at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler.internalExecute(TransactionalProtocolOperationHandler.java:247)
>         at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler.doExecute(TransactionalProtocolOperationHandler.java:185)
>         at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:138)
>         at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:134)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:360)
>         at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:81)
>         at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:157)
>         at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2$1.run(TransactionalProtocolOperationHandler.java:153)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$2.execute(TransactionalProtocolOperationHandler.java:153)
>         at org.jboss.as.protocol.mgmt.AbstractMessageHandler$ManagementRequestContextImpl$1.doExecute(AbstractMessageHandler.java:363)
>         at org.jboss.as.protocol.mgmt.AbstractMessageHandler$AsyncTaskRunner.run(AbstractMessageHandler.java:472)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
>         at org.jboss.threads.JBossThread.run(JBossThread.java:320)
> {code}
> This error will usually only happen for 2 out of the 4 identically configured slaves, and seems to happen randomly, although frequently enough.



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)