You are right, adding our own VE is a working solution for us.

Just a few more thoughts:
I have no idea about additional VEs, but the default ones are about 20 for classes (where more than 1/2 is final) and 5 for interfaces.

As for the bag: Why not? The runtime implementation of the bag might well be ordered and/or allow index based access using the List interface.
IMHO "Set" or "Bag" semantics do not prohibit index based access - and if it is available I would love to see the index in the constraints!

And what is the "intention" of the user if he declares "Object"? When I follow your argumentation he has no intention at all in this case and should not expect any VE to be applied!? IMHO everything else would be inconsistent.