[hibernate-dev] HSEARCH-598 (was: Beginner's tasks for Hibernate Search and Hibernate OGM)

[Guillaume Smet]

Hi Sanne,

On Thu, Mar 1, 2012 at 1:09 PM, Sanne Grinovero <sanne at hibernate.org> wrote:
> I'm aware of the issue, and I wouldn't mind some help on it: it would
> be better to fix the MassIndexer design to use a limited set of
> threads than to expose more metadata information, don't you agree?
>
> The reason for me to not have given a high priority to HSEARCH-598 are
>  1) there are easy workarounds

I agree *IF* we provide an API to get the root indexed entities. The
most simple workaround so far is to work entity by entity and it would
be nice to be able to do so easily.

This is how we do it currently:
for (Class<?> clazz :
getIndexedRootEntities(fullTextEntityManager.getSearchFactory(),
Object.class)) {
				LOGGER.debug(String.format("Reindexing %1$s.", clazz));
				ProgressMonitor progressMonitor = new ProgressMonitor();
				Thread t = new Thread(progressMonitor);
				t.start();
				MassIndexer indexer = fullTextEntityManager.createIndexer(clazz);
				indexer.batchSizeToLoadObjects(batchSize)
						.threadsForSubsequentFetching(fetchingThreads)
						.threadsToLoadObjects(loadThreads)
						.cacheMode(CacheMode.NORMAL)
						.progressMonitor(progressMonitor)
						.startAndWait();
				progressMonitor.stop();
				t.interrupt();
				LOGGER.debug(String.format("Reindexing %1$s done.", clazz));
			}

with getIndexedRootEntities(...):
        /**
	 * @see MassIndexerImpl#toRootEntities
	 */
	protected Set<Class<?>> getIndexedRootEntities(SearchFactory
searchFactory, Class<?>... selection) {
		if (searchFactory instanceof SearchFactoryImplementor) {
			SearchFactoryImplementor searchFactoryImplementor =
(SearchFactoryImplementor) searchFactory;
			
			Set<Class<?>> entities = new HashSet<Class<?>>();
			
			// first build the "entities" set containing all indexed subtypes
of "selection".
			for (Class<?> entityType : selection) {
				Set<Class<?>> targetedClasses =
searchFactoryImplementor.getIndexedTypesPolymorphic(new Class[] {
entityType });
				if (targetedClasses.isEmpty()) {
					String msg = entityType.getName() + " is not an indexed entity or
a subclass of an indexed entity";
					throw new IllegalArgumentException(msg);
				}
				entities.addAll(targetedClasses);
			}
			
			Set<Class<?>> cleaned = new HashSet<Class<?>>();
			Set<Class<?>> toRemove = new HashSet<Class<?>>();
			
			//now remove all repeated types to avoid duplicate loading by
polymorphic query loading
			for (Class<?> type : entities) {
				boolean typeIsOk = true;
				for (Class<?> existing : cleaned) {
					if (existing.isAssignableFrom(type)) {
						typeIsOk = false;
						break;
					}
					if (type.isAssignableFrom(existing)) {
						toRemove.add(existing);
					}
				}
				if (typeIsOk) {
					cleaned.add(type);
				}
			}
			cleaned.removeAll(toRemove);
			
			if (LOGGER.isInfoEnabled()) {
				LOGGER.info("Targets for indexing job: {}", cleaned);
			}
	
			return cleaned;
		} else {
			throw new IllegalArgumentException("searchFactory should be a
SearchFactoryImplementor");
		}
	}

>  2) I have no tests
>
> Writing an automated test is in this case I think 90% of the
> complexity; if you would like to do that, I think I could fix it
> quickly or at least propose many possible solutions.
> Ideally for this to be included in 4.1.0.Final we should finish it
> next week.. do you think you could help with it?

Not at the moment. You might have seen that I'm far less active than
last month, mostly because I have a lot of work at my company - and at
the moment on a project using Solr instead of Hibernate Search - but
still based on Hibernate. We might be able to help on this front and
others later this year as Hibernate Search is an important part of our
work.

-- 
Guillaume