Testing I have done has shown the use of optimistic locking to be too buggy. As you have demonstrated concurrent transactions start to compete using the same ids.
The existing mechanism using pessimistic locking is the way to go.