[keycloak-user] Mixing Identity Broker with Federated provider

Hi, Is it possible to use Federated LDAP/AD for importing accounts but use OAuth broker for authenticating them users? I am importing users from local AD server which sets up the users upn as the login name and email address. I have then setup Azure AD using OIDC identity broker and setup a "Username Template Importer" to use the same upn attribute from the OAuth claims so that the username and email match that of the imported LDAP accounts. But when I login using Azure AD, keycloak complains of an existing username already present. Is there a way I can have Azure AD and Federated LDAP imports work together so that it assumes the same identity? Thanks.