Original problem was reported to batch fetching.
Even though IdentityMap is used harder in another place like at flushing data - IdentityMap is a buttle neck in batch fetching. Creating new HashSet with big content really takes a lot of CPU time, especially if Java provides unlucky implementation of HashSet. It also produces a lot of small temporary objects. But both "the unlucky implementation" and Oracle/Sun one's HashSet are not implemented in this way - just an adapter is created that works over the original collection.
I beleive IdentityMap.entrySet() should be fixed this way finally.