The Java Security Manager makes a lot of sense also on the server side when you are building component based software and want for instance to ensure that some components are eligible to access some data and others are not.
If you can ensure that most components are not able to access certain sensitive data then you can skip them in security audits and that’s a great win!