Do you ever serialize and deserialize entities?
As AbstractLazyInitializer is not serializable, it forgets about the initial allowLoadOutsideTransaction = true.