I am not overly concerned with the mapping being different between simple and composite keys. trying to list the other differences, the mapping would be different also between a regular Collection or Set from a List or Map (due to their index / key).

I don't think justifying the behaviour difference based on the split element collection vs association really holds. In many ways they are the same thing. The lifecycle of the former is more constrained that's all.

So I think my heart leans towards a unified representation (and code).