[hibernate-dev] [Search] Lazy initialization vs HSearch - Round 4

Hi, Well, this time, I'm not sure we will be able to win the round by ourselves so I'm asking for advice and guidance... Following our recent patch submissions on the subject, we have one problem left: when we're doing a mass import (note: this mass import is really part of the application so we can't just disable Hibernate Search during it), we have the following pattern: Transaction { for (items partitioned by 100) { session.flush(); session.clear(); do work for these 100 items(); } } -> indexing is done at the end of the transaction by Hibernate Search automatically. Note that we really need to flush the session otherwise it really becomes too slow. detached from the session so the LuceneWorks executed when the transaction is commited are working on detached entities. Are we correct on this? The fact is that following the addition of fetch = FetchType.LAZY in certain relations, we start to have the following problems. Both are related though. I'm pretty sure we missed something as the problem seems obvious but isn't reproducible without all the steps we have in our import. We thought it might also be related to the fact that some entities might come from the cache. * Problem 1: it doesn't throw an exception but it doesn't sound right * This code in AbstractDocumentBuilder.appendContainedInWorkForInstance() is triggered as the session of the PersistentSet it's working on is null: try { collection = getActualCollection( member, value ); collection.size(); //load it } catch ( Exception e ) { if ( e.getClass().getName().contains( "org.hibernate.LazyInitializationException" ) ) { /* A deleted entity not having its collection initialized * leads to a LIE because the collection is no longer attached to the session * * But that's ok as the collection update event has been processed before * or the fk would have been cleared and thus triggering the cleaning */ collection = null; } } -> the collection is then completely ignored but shouldn't be as we are not working on a deleted entity. * Problem 2: this time, the exception is not caught and it breaks our mass import * When Hibernate Search tries to index an entity, it throws a LazyInitException in HibernateStatelessInitializer.unproxy() as, one more time, the session of the lazy proxy is null: it triggers the throw new LazyInitializationException( "could not initialize proxy - no Session" ); in AbstractLazyInitializer.initialize(). The fact is that the trick we added in the unproxy() method of HibernateLoadingSessionInitializer recently (see https://github.com/openwide-java/hibernate-search/commit/2f7844857bfafb95... and next commit on this file) cannot work as we don't have a session in Hibernate*Stateless*Initializer. We're kinda stuck here and I'm pretty sure we're missing something. Any idea? Thanks in advance. -- Guillaume