do you use UserGroupCallback? If so it could be that callback does remove users that it does not consider as valid. Best would be to debug it and see why potential owners are removed when task is created. Additionaly can you check (to cofirm) that there are no potential owners stored in db for that task?
HTH