[wildfly-dev] WFCORE-1157

[Kabir Khan]

Jeff and I were chatting about https://issues.jboss.org/browse/WFCORE-1157 last week. There is currently a PR (https://issues.jboss.org/browse/WFCORE-1157?devStatusDetailDialog=pullrequest) to allow listening on the ControlledProcessState state changes. This is done via users registering NotificationHandlers on the runtime-configuration-state.

Since the notification handlers are executed asynchronously, there is no guarantee that e.g. on a stop that the notification handler is triggered for the 'stopping' and 'stopped' (the PR introduces this latter state) state changes since the server may be down before this happens. The PR works around this by making the controller execute the runtime-configuration-state handlers synchronously. However, this then means that the standard notifications and the runtime-configuration-state notifications end up being in separate streams, so that the 'stopping' handler may be invoked upon before the standard/async notification handlers reflecting earlier changes.

In fact looking at this a bit closer, the NonBlockingNotificationSupport class uses a thread pool with several threads. This means that for the  standard async notifications, it is very likely that the handler for notification1 gets invoked before notification2's handler, but is is _not_ guaranteed. If the thread processing notification1 is paused for whatever reason, notification2 may end up being handled first. Should we change the executor in NonBlockingNotificationSupport to be a single thread executor?

Jeff also suggested, perhaps keeping the runtime-configuration-state notifications as asynchronous, but to add some constructs to make sure that these always get executed before server shutdown. This would keep the functionality from this PR where the notifications are always invoked, and also make sure that the order is preserved.