[keycloak-user] Ldap sync

I have setup my keycloak with a read only LDAP User Federation Provider and set it up to sync periodically, but for some reason only some users are not syncing , not all users , only some. I tried to trigger the sync with the "Synchronize all users" but no luck. The only way it worked is by completely removing the provider and adding it again. which is not a great solution . Have anyone seen this ? is there a way to fix it ? Thanks