When reference cached entities are fetched from cache, both a postLoad and afterInitialize calls are made which add up a lot of CPU time.

Several methods on DefaultLoadEventListener can be refactored to more easily be inlined, the LOG.isTracingEnabled call can be cached.