Another update on this since Brian noticed on Friday that the cloud tests were only actually testing the main branch of WildFly and not the pull request contents. This is because the 'Cloud Tests Trigger' workflow was handling the 'pull_request_target' event since it needs access to a PAT stored in the secrets to do the remote dispatch to the job in the wildfly-cloud-tests repository. 

However, 'pull_request_target' workflows do not have access to the sha of the merge commit from the PR into WildFly main. That is only available in workflows handling 'pull_request' which in turn cannot access secrets.

So I have split the trigger job into two.
* cloud-test-pr-trigger.yml (i.e. 'Cloud Tests Trigger'). This now handles the 'pull_request' event instead. It reads the sha of the merge commit, and uploads that as an internal artifact.
* cloud-test-pr-workflow-run.yml (i.e. 'Cloud Tests Workflow Run'). This handles the 'workflow_run' event which has access to secrets, and runs when 'Cloud Tests Trigger' has completed. It gets information about sha from the artifact the other workflow updated, and then does the work to do the remote dispatch.

Since 'workflow_run' jobs seem to be silent (they don't appear in the Checks box in the PR), from a user perspective things remain much the same. So for the cloud tests we only see the same two entries as before on the pull request. The new 'Cloud Tests Workflow Run' workflow still appears in the Actions pane of the repository so we can see if something is going wrong.

To recap with what we see on the PR:
* Cloud Tests Trigger - returns right away after reading the information, and should always pass. If there is a failure, this is the job to rerun.
* Cloud Tests Remote Run - contains the status of the remote job

image.png

Since 'Cloud Tests Remote Run' didn't appear until the remote job in the cloud tests repository reports that the job is pending (and later the final status), I added similar reporting to the new 'Cloud Tests Workflow Run' workflow so it doesn't fail silently if the remote dispatch, which now runs in the background, fails.

So the statuses, in the expected order, of the 'Cloud Tests Remote Run' check are:
* Pending - Invoking remote job - Set by 'Cloud Tests Remote Run' before attempting the remote dispatch but after reading the information from the artifact. This information is needed to know where to report back
** Failure - The attempt to start the remote job failed, or was cancelled - Set by 'Cloud Tests Remote Run' if the remote dispatch failed. If this happens the remote job will not run, so this is a final status.
* Pending - Running Tests - The job is running in the wildfly-cloud-tests repository (same as before) 
* Success/Failure - The final result of the job  in the wildfly-cloud-tests repository (same as before)

Thanks,

Kabir