[JBoss JIRA] (WFLY-3549) Deadlock during shutdown
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/WFLY-3549?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on WFLY-3549:
-----------------------------------------------
Mike McCune <mmccune(a)redhat.com> changed the Status of [bug 1213817|https://bugzilla.redhat.com/show_bug.cgi?id=1213817] from MODIFIED to POST
> Deadlock during shutdown
> ------------------------
>
> Key: WFLY-3549
> URL: https://issues.jboss.org/browse/WFLY-3549
> Project: WildFly
> Issue Type: Bug
> Affects Versions: 8.1.0.Final
> Reporter: Dan Berindei
> Assignee: David Lloyd
> Fix For: 8.2.0.Final
>
>
> This deadlock appeared in an Arquillian test:
> {noformat}
> Found one Java-level deadlock:
> =============================
> "undefined":
> waiting to lock monitor 0x00007f67a421bfa8 (object 0x00000000e0700480, a org.jboss.as.threads.ScheduledThreadPoolService),
> which is held by "MSC service thread 1-2"
> "MSC service thread 1-2":
> waiting for ownable synchronizer 0x00000000e0700618, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
> which is held by "undefined"
> Java stack information for the threads listed above:
> ===================================================
> "undefined":
> at org.jboss.as.threads.ScheduledThreadPoolService$ExecutorImpl.terminated(ScheduledThreadPoolService.java:121)
> - waiting to lock <0x00000000e0700480> (a org.jboss.as.threads.ScheduledThreadPoolService)
> at java.util.concurrent.ThreadPoolExecutor.tryTerminate(ThreadPoolExecutor.java:704)
> at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1006)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1163)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> at org.jboss.threads.JBossThread.run(JBossThread.java:122)
> "MSC service thread 1-2":
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000e0700618> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
> at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:214)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:290)
> at java.util.concurrent.ThreadPoolExecutor.interruptIdleWorkers(ThreadPoolExecutor.java:781)
> at java.util.concurrent.ThreadPoolExecutor.tryTerminate(ThreadPoolExecutor.java:695)
> at java.util.concurrent.ThreadPoolExecutor.shutdown(ThreadPoolExecutor.java:1397)
> at java.util.concurrent.ScheduledThreadPoolExecutor.shutdown(ScheduledThreadPoolExecutor.java:759)
> at org.jboss.as.threads.ManagedScheduledExecutorService.internalShutdown(ManagedScheduledExecutorService.java:53)
> at org.jboss.as.threads.ScheduledThreadPoolService.stop(ScheduledThreadPoolService.java:67)
> - locked <0x00000000e0700480> (a org.jboss.as.threads.ScheduledThreadPoolService)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
> at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Found 1 deadlock.
> {noformat}
> Looks like two MSC service threads exited and tried to terminate the thread pool at the same time. And because the MSC threads are not daemon threads, the entire JVM hangs and blocks the Arquillian test that was waiting for the container to shut down.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 1 month
[JBoss JIRA] (WFCORE-632) Subsystem deployment undeployed at startup
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/WFCORE-632?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on WFCORE-632:
------------------------------------------------
Mike McCune <mmccune(a)redhat.com> changed the Status of [bug 1271673|https://bugzilla.redhat.com/show_bug.cgi?id=1271673] from MODIFIED to POST
> Subsystem deployment undeployed at startup
> ------------------------------------------
>
> Key: WFCORE-632
> URL: https://issues.jboss.org/browse/WFCORE-632
> Project: WildFly Core
> Issue Type: Bug
> Components: Domain Management
> Affects Versions: 1.0.0.Beta3, 1.0.0.Beta4
> Reporter: Stan Silvert
> Assignee: Brian Stansberry
> Labels: affects_elytron
> Fix For: 1.0.0.Beta4
>
>
> {noformat}
> @BrianStansberry The "mixed approach" doesn't seem to work any more. https://developer.jboss.org/wiki/ExtendingAS7
> @BrianStansberry I'm using this to deploy keycloak WAR from the subsystem.
> @BrianStansberry As soon as the server is started, something is calling stopService() on the Keycloak WAR.
> Tomaz Cerar
> 1:15 PM
> @StanSilvert are you still working on AS7? https://docs.jboss.org/author/display/WFLY8/Extending+WildFly+8
> Stan Silvert
> 1:16 PM
> @ctomc No. this is WildFly 9. It still works on WildFly 8.
> Tomaz Cerar
> 1:16 PM
> ah the war deployment, that could be regression
> Brian Stansberry
> 1:17 PM
> @ctomc how so?
> Stan Silvert
> 1:17 PM
> FYI. I did a Thread.dumpStack() and got this:
> 13:11:35,173 ERROR [stderr] (MSC service thread 1-3) java.lang.Exception: Stack trace
> 13:11:35,173 ERROR [stderr] (MSC service thread 1-3) at java.lang.Thread.dumpStack(Thread.java:1329)
> 13:11:35,174 ERROR [stderr] (MSC service thread 1-3) at org.wildfly.extension.undertow.Host.unregisterDeployment(Host.java:168)
> 13:11:35,176 ERROR [stderr] (MSC service thread 1-3) at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.stopContext(UndertowDeploymentService.java:
> 113)
> 13:11:35,179 ERROR [stderr] (MSC service thread 1-3) at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.stop(UndertowDeploymentService.java:100)
> 13:11:35,181 ERROR [stderr] (MSC service thread 1-3) at org.jboss.msc.service.Serv
> iceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
> 13:11:35,184 ERROR [stderr] (MSC service thread 1-3) at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
> 13:11:35,186 ERROR [stderr] (MSC service thread 1-3) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 13:11:35,188 ERROR [stderr] (MSC service thread 1-3) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 13:11:35,190 ERROR [stderr] (MSC service thread 1-3) at java.lang.Thread.run(Thread.java:745)
> Hide full text
> Tomaz Cerar
> 1:18 PM
> @BrianStansberry just reffering that war deployment from subsystem should still work
> 1:19 PM
> Jason Greene joined the room.
> Tomaz Cerar
> 1:19 PM
> @StanSilvert what happens? as that thread dump makes no sense
> Stan Silvert
> 1:21 PM
> http://pastebin.com/xQ2DNzEe
> The WAR is simply undeployed as soon as WildFly finishes startup.
> Brian Stansberry
> 1:22 PM
> @StanSilvert can you give me a link to your code that's doing the deploy stuff?
> Stan Silvert
> 1:22 PM
> just a sec
> Stan Silvert
> 1:24 PM
> https://github.com/keycloak/keycloak/tree/master/integration
> The code that creates the operation is https://github.com/keycloak/keycloak/blob/master/integration
> AuthServerUtil ^^^
> click on the second link
> 1:27 PM
> Darran Lofthouse left the room.
> Brian Stansberry
> 1:27 PM
> @StanSilvert are you doing overlay stuff? I see some code there re: overlays
> Stan Silvert
> 1:28 PM
> Yes, but not in this instance.
> Brian Stansberry
> 1:28 PM
> ok. I'm asking just because that would add more complexity, better scope for some service dependency going missing, triggering stop
> Stan Silvert
> 1:29 PM
> For this simple case there are no overlays.
> Brian Stansberry
> 1:29 PM
> @StanSilvert interesting, your log looks like it's a true undeploy op, not just a service stop
> Tomaz Cerar
> 1:30 PM
> @BrianStansberry should we use 6.x.0 or 6.x.latest for tranformers testing?
> Tomaz Cerar goes get some diner
> Stan Silvert
> 1:30 PM
> @BrianStansberry If that's the case then maybe some condition is triggering my own undeploy operation.
> @BrianStansberry I'll look into that and see what I find.
> Brian Stansberry
> 1:31 PM
> @ctomc 6.x.0 is ok by me; chasing CPs is too much hassle
> @StanSilvert note this:
> 13:11:35,294 INFO [org.jboss.as.server] (DeploymentScanner-threads - 1) WFLYSRV0009: Undeployed "main-auth-server.war" (runtime-name: "main-auth-server.war")
> the thread -- DeploymentScanner-threads - 1
> looks like your deployment is exposed to the scanner?
> oh, here's a guess!
> the scanner sees "persistent" => false and treats it as under scanner control
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 1 month
[JBoss JIRA] (WFCORE-547) Unable to overlay resource in subunit/zip/jar file
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/WFCORE-547?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on WFCORE-547:
------------------------------------------------
Mike McCune <mmccune(a)redhat.com> changed the Status of [bug 1147352|https://bugzilla.redhat.com/show_bug.cgi?id=1147352] from MODIFIED to POST
> Unable to overlay resource in subunit/zip/jar file
> --------------------------------------------------
>
> Key: WFCORE-547
> URL: https://issues.jboss.org/browse/WFCORE-547
> Project: WildFly Core
> Issue Type: Bug
> Components: Server
> Affects Versions: 1.0.0.Alpha18
> Reporter: Bartosz Baranowski
> Assignee: Stuart Douglas
> Fix For: 1.0.0.Beta1
>
>
> It is possible to overlay resource directly in deployed jar, however, same resource can not be properly overlayed if said jar is deployed as part of ear ( possibly war as well ).
> Example:
> jar (xxx.jar):
> /org/test/ejb/EJBImpl.class
> /org/test/ejb/resourceToOverlya.txt
> ear(yyy.ear) - war seems to be the subject of this as well
> /xxx.jar
> Overlay #1( over jar)
> overlay target: xxx.jar/org/test/ejb/resourceToOverlya.txt
> Result: proper deployment(EJB is deployed), resource has been overlayed
> Overlay #2( over jar inside ear )
> overlay target: org/test/ejb/resourceToOverlya.txt
> Result: bad deployment, no EJB deployed, overlay target has been overlayed
> Reason of such outcome is that VFS.exists() does not recognize JavaZipFileSystem$ZipNode/Entry as zipped archive and does not dive into children.
> So what happens in here is as follows. When EAR is deployed OverlayProcessor kicks in at:
> https://github.com/wildfly/wildfly-core/blob/master/server/src/main/java/...
> For this example ear/jar pair, first pair of values:
> mountPoint: /content/yyy.ear/xxx.jar/org/test/ejb/resourceToOverlya.txt
> parent: /content/yyy.ear/xxx.jar/org/test/ejb
> Outcome of exists() call == false, should be true.
> this will iterate up to parent == /content/yyy.ear/xxx.jar
> At this point, 'createParents' contain some entries and
> https://github.com/wildfly/wildfly-core/blob/master/server/src/main/java/...
> will create tmp/virtual mounts for:
> - /content/yyy.ear/xxx.jar/org/test/ejb
> - /content/yyy.ear/xxx.jar/org/test
> - /content/yyy.ear/xxx.jar/org
> What it will essentially do is create overlay for whole 'org' package! The VirtualFile.getChildren() will return tmp mounted VirtualFile for 'org' entry: https://github.com/jbossas/jboss-vfs/blob/master/src/main/java/org/jboss/...
> There is part of code which should remove 'submount' if there is directory entry: https://github.com/jbossas/jboss-vfs/blob/master/src/main/java/org/jboss/... however ( if I remember correctly) 'org' tmp mount ends up twice in 'submounts'
> When VirtualFile.getChildren is called for 'this' == /content/yyy.ear/xxx.jar ( in this case it is properly removed) and when 'this' == /content/yyy.ear/xxx.jar/org - in which case it derails getChildren and diverts into tmp mount.
> Now about https://github.com/jbossas/jboss-vfs/blob/master/src/main/java/org/jboss/...
> What happens here, for ear deployment - it dives recursively into 'find' until 'target' is equal to '/content/yyy.ear/xxx.jar' at this target.getParent() == 'mountPoint' == '/content/yyy.ear'.
> At this point 'find' will get result ZipNode == /content/yyy.ear and fetch child 'xxx.jar' and pop back up one notch in the call stack. However the 'xxx.jar' entry does not have any children, hence there is no way to fetch next child in line 'org'.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 1 month
[JBoss JIRA] (WFCORE-761) Not possible to overlay non existing file in WAR
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/WFCORE-761?page=com.atlassian.jira.plugin... ]
RH Bugzilla Integration commented on WFCORE-761:
------------------------------------------------
Mike McCune <mmccune(a)redhat.com> changed the Status of [bug 1147352|https://bugzilla.redhat.com/show_bug.cgi?id=1147352] from MODIFIED to POST
> Not possible to overlay non existing file in WAR
> ------------------------------------------------
>
> Key: WFCORE-761
> URL: https://issues.jboss.org/browse/WFCORE-761
> Project: WildFly Core
> Issue Type: Bug
> Components: Server
> Reporter: Bartosz Baranowski
> Assignee: Dimitris Andreadis
> Priority: Critical
>
> It is either bug in how deployments are treated or how overlay/vfs work.
> Steps to reproduce:
> 1. deploy undexploded war with jar inside
> 2. add overlay that will add non existing file in jar
> Result: exception:
> Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: JBAS018776: Failed to get content for deployment overlay WEB-INF/lib/overlayed.jar//META-INF/x/file.txt at WEB-INF/lib/overlayed.jar//META-INF/x/file.txt
> Caused by: java.io.FileNotFoundException: /content/shell.war/WEB-INF/lib/overlayed.jar/META-INF/x/file.txt"}}
> at org.jboss.as.test.integration.management.ManagementOperations.executeOperation(ManagementOperations.java:67)
> at org.jboss.as.test.integration.management.ManagementOperations.executeOperation(ManagementOperations.java:37)
> at org.jboss.as.test.integration.deployment.deploymentoverlay.jar.OverlayUtils.setupOverlay(OverlayUtils.java:76)
> at org.jboss.as.test.integration.deployment.deploymentoverlay.war.OverlayNonExistingResourceTestCase.testOverlay(OverlayNonExistingResourceTestCase.java:67)
> Expectation:
> should work. It actually does work, if war is really exploded or
> 'if(exploded)' part in overlay is removed from overlay processor and everything is handled via: https://github.com/stuartwdouglas/wildfly-core/blob/a75af9118c4062fafb899...
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)
8 years, 1 month