[JBoss JIRA] (WFLY-6895) TimerService problem(duplicated resource)
by Tom Arnold (JIRA)
[ https://issues.jboss.org/browse/WFLY-6895?page=com.atlassian.jira.plugin.... ]
Tom Arnold commented on WFLY-6895:
----------------------------------
There has been a pull request sitting out there since July 2016. [~jason.greene] could someone on the Wildfly team please review this?
I think I am running into this on 10.1.0.Final. The fix in the PR appears to resolve the issue.
This problem seems similar to WFLY-4908; both are cases where redeployment of dependencies break their dependents.
> TimerService problem(duplicated resource)
> -----------------------------------------
>
> Key: WFLY-6895
> URL: https://issues.jboss.org/browse/WFLY-6895
> Project: WildFly
> Issue Type: Bug
> Affects Versions: 10.0.0.Final
> Environment: Two standalone instances connected into a cluster.
> *Master WildFly*
> {code}
> standalone.bat -c standalone-full-ha.xml -Djboss.node.name=master
> {code}
> *Slave WildFly*
> {code}
> standalone.bat -c standalone-full-ha.xml -Djboss.node.name=slave -Djboss.socket.binding.port-offset=100
> {code}
> Both instances has the same singleton policy defined in singleton-full-ha.xml:
> {code}
> <singleton-policy name="scada-singleton" cache-container="server">
> <simple-election-policy>
> <name-preferences>master</name-preferences>
> </simple-election-policy>
> </singleton-policy>
> {code}
> Reporter: Artur Kowalczyk
> Assignee: Tomasz Adamski
>
> I found a problem with TimerService that occurs when your application is configured as a singleton deployment.
> *Test Case*
> # Start master node, the app is also started - OK
> # Start slave node, the app is deployed but bot stared - OK
> # Stop master node, the app is started on slave - OK
> # Start master node, there is a preference for node name so the app will be started again on master and stopped in slave - OK
> # Stop master node, the app should be started again on slave but an exception occur. - ERROR
> {code}
> 09:50:42,115 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-6) MSC000001: Failed to start service jboss.deployment.subunit."wildfly-ejb-in-ear.ear"."wildfly-ejb-in-ear-ejb.jar".INSTALL: org.jboss.msc.service.StartException in service jboss.deployment.subunit."wildfly-ejb-in-ear.ear"."wildfly-ejb-in-ear-ejb.jar".INSTALL: WFLYSRV0153: Failed to process phase INSTALL of subdeployment "wildfly-ejb-in-ear-ejb.jar" of deployment "wildfly-ejb-in-ear.ear"
> at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:154)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)
> at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: WFLYEJB0086: Failed to install management resources for TimerEJB
> at org.jboss.as.ejb3.deployment.processors.EjbManagementDeploymentUnitProcessor.deploy(EjbManagementDeploymentUnitProcessor.java:82)
> at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:147)
> ... 5 more
> Caused by: java.lang.IllegalStateException: WFLYCTL0075: Duplicate resource timer-service
> at org.jboss.as.controller.registry.AbstractModelResource$DefaultResourceProvider.register(AbstractModelResource.java:290)
> at org.jboss.as.controller.registry.AbstractModelResource.registerChild(AbstractModelResource.java:169)
> at org.jboss.as.server.deployment.DeploymentResourceSupport.register(DeploymentResourceSupport.java:322)
> at org.jboss.as.server.deployment.DeploymentResourceSupport.registerDeploymentSubResource(DeploymentResourceSupport.java:219)
> at org.jboss.as.ejb3.deployment.processors.EjbManagementDeploymentUnitProcessor.installManagementResource(EjbManagementDeploymentUnitProcessor.java:119)
> at org.jboss.as.ejb3.deployment.processors.EjbManagementDeploymentUnitProcessor.deploy(EjbManagementDeploymentUnitProcessor.java:79)
> ... 6 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 2 months
[JBoss JIRA] (WFCORE-2793) Consolidate scheduled thread pool usage
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-2793?page=com.atlassian.jira.plugi... ]
Brian Stansberry updated WFCORE-2793:
-------------------------------------
Summary: Consolidate scheduled thread pool usage (was: Get rid of the "DeploymentScanner Threads" pool)
> Consolidate scheduled thread pool usage
> ---------------------------------------
>
> Key: WFCORE-2793
> URL: https://issues.jboss.org/browse/WFCORE-2793
> Project: WildFly Core
> Issue Type: Bug
> Components: Deployment Scanner, Domain Management
> Reporter: Brian Stansberry
> Assignee: Brian Stansberry
>
> Look into whether we can consolidate thread usage for scheduled tasks.
> 1) Get rid of the "DeploymentScanner Threads" pool
> The deployment scanner can use the kernel scheduled executor. I'm a bit reluctant in general to allow use of this pool by subsystems as scheduled executors have a fixed size pool, so arbitrary usage by subsystems can result in tying up all the threads doing long running tasks and undesirable behavior. But deployment-scanner is "kernel-ish enough" that I think it's ok.
> This will remove 2 threads.
> The server scheduled executor has 4 threads which is actually pretty high given the very limited usage of it. So I'll consider narrowing it down.
> The big problem with using the server scheduled executor is tying up its threads long running tasks, which the scanner does do. What i'll probably do is just use the scheduled executor to trigger a task which then submits a task on the main ServerService thread pool (which is unlimited in size.)
> Perhaps I'll abstract this kind of usage pattern into a service, and make that service a generally available kernel capability.
> 2) Get rid of the "ServerDeploymentRepository-temp-threads" pool
> This one is used by DeploymentMountProvider to do file cleanup. Similar discussion applies.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 2 months
[JBoss JIRA] (WFCORE-2793) Get rid of the "DeploymentScanner Threads" pool
by Brian Stansberry (JIRA)
[ https://issues.jboss.org/browse/WFCORE-2793?page=com.atlassian.jira.plugi... ]
Brian Stansberry commented on WFCORE-2793:
------------------------------------------
Skipping queued content repo cleaner tasks is fine; the repo cleaner can run the next time the server is booted.
ResponseAttachmentInputStreamSupport.shutdown() cleans up the input streams in server stop scenarios. The scheduled task is to clean them up if the server isn't stopping. So not executing queued tasks is fine.
TempFileProvider.deleteTask schedules an attempt to delete a file if it cannot be deleted immediately. Since the threads in the pool are non-daemon there is no guarantee this will happen following shutdown (probably won't). *However, following a reload, where the daemon threads live on, perhaps not running these would have negative effects; i.e. the reload eliminates whatever was preventing the initial delete, so then later the DeleteTask can succeed.*
The deployment scanner uses its executor as follows:
DefaultDeploymentOperations -- here it's not using it for scheduled tasks, and the non-scheduled ServerService.ServerExecutorService could be used. The pool is used to create a LocalModelControllerClient. So no relevance to scheduled task shutdown.
FileSystemDeploymentService -- the main thing...
BootTimeScannerDeployment.deploy(...) uses it to execute an async task, but it could probably be given ServerService.ServerExecutorService in its constructor and the Executor param dropped. Or FilesystemDeploymentService can pass in the ServerService.ServerExecutorService.
FilesystemDeploymentService.startScan() executes DeploymentScanRunnable. There is no reason to run one of these if from the scheduled executor queue, as it will just see that 'scanEnabled != true' and return.
singleScan() executes ManualScan. This is not a delayed task, and ServerService.ServerExecutorService could handle it.
scan() submits another DeploymentScanRunnable if the first run returned Status.RETRY, i.e. there was incompletely copied content in the deployments dir. This basically is to try again quickly in order to see if the copy is complete. There is no reason this needs to run if the scheduled executor is shut down. The server is being stopped; there's no point trying to deploy to it.
*Summary*
The only negative I see to simply discarding queued tasks on scheduled executor shutdown is the case of a reload, where TempFileProvider.deleteTask currently may eventually run and do some cleanup.
> Get rid of the "DeploymentScanner Threads" pool
> -----------------------------------------------
>
> Key: WFCORE-2793
> URL: https://issues.jboss.org/browse/WFCORE-2793
> Project: WildFly Core
> Issue Type: Bug
> Components: Deployment Scanner, Domain Management
> Reporter: Brian Stansberry
> Assignee: Brian Stansberry
>
> Look into whether we can consolidate thread usage for scheduled tasks.
> 1) Get rid of the "DeploymentScanner Threads" pool
> The deployment scanner can use the kernel scheduled executor. I'm a bit reluctant in general to allow use of this pool by subsystems as scheduled executors have a fixed size pool, so arbitrary usage by subsystems can result in tying up all the threads doing long running tasks and undesirable behavior. But deployment-scanner is "kernel-ish enough" that I think it's ok.
> This will remove 2 threads.
> The server scheduled executor has 4 threads which is actually pretty high given the very limited usage of it. So I'll consider narrowing it down.
> The big problem with using the server scheduled executor is tying up its threads long running tasks, which the scanner does do. What i'll probably do is just use the scheduled executor to trigger a task which then submits a task on the main ServerService thread pool (which is unlimited in size.)
> Perhaps I'll abstract this kind of usage pattern into a service, and make that service a generally available kernel capability.
> 2) Get rid of the "ServerDeploymentRepository-temp-threads" pool
> This one is used by DeploymentMountProvider to do file cleanup. Similar discussion applies.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
9 years, 2 months