I finally found and fixed a (if not the) problem. In the global permissions, I've replaced `assigned-jira` to `users` as the default group associated to new registered users.
Now there is still something fishy as it "takes a while" for a new user to receive the right permissions after creation. 10 mins to 30 mins in my experimentations.
That sucks but after this waiting period, the "Create" Issue button does appear.

I'm going to close that issue.