In the installer we're currently doing this every time the server starts as we're using an in-memory database so that is cleaned every time the server goes down. If you use a more permanent database, you should only insert the valid users / groups once. Also notice that (1) this data is not stored in memory, only in the database, so storing a few thousand users shouldn't be an issue and (2) a user or group is just a String that represents its id, so the data itself is also very limited.
So I don't think it's as bad as you make it sound ;) But we'll improve this asap as well ...
Kris