[keycloak-user] Force certain realm users to login via IDP

We are looking into using IDP (Azure AD) for login. Some users (admins) will then authenticate there. The need for this is that Keycloak admins (user management in certain realm) will need to authenticate via two factor because of company policies. So I've already setup a working integration with AD. The problem now is that pre-existing users that already had a login and password in Keycloak must no longer be able to use login/password. This is to force IDP (two factor) login. I've tried to "Disable Credentials" for "password" for such a user but still he could login. I'm thinking of a solution where we script a custom browser flow action where we check is the user is a admin and then denies him if using password. Any thoughts or suggestions? Regards Tim