Public clients rely on the redirect uri to prevent malicious clients from obtaining a token via the SSO session. As there is no redirect uri in direct grant there would no longer be a mechanism to prevent malicious clients. I know there's some level of protection in CORS, but IMO that on its own is not sufficient. A public client should require both redirect uri and CORS protection if it wants to utilize the SSO session.
It would also be inconsistent with the OIDC spec. There's nothing there about using SSO with direct grant. SSO is only available via web redirect flows and there's good reasons for that.