[hibernate-issues] [Hibernate-JIRA] Commented: (HSEARCH-601) Allow MassIndexer to recover gracefully when individual objects won't index

[Sanne Grinovero (JIRA)]

    [ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=38878#action_38878 ] 

Sanne Grinovero commented on HSEARCH-601:
-----------------------------------------

so you're using a http://docs.jboss.org/hibernate/search/3.3/reference/en-US/html_single/#d0e1811 to show the errors on the UI? or would it work for you? We've been discussing this, and after a quick POC it seems the way to go.
Errors would be handled by the error handler; Fail-fast will still be the default (handling the first error only and then abort), and a new option can let you "fail-never" and receive all errors in whatever error handler you might have configured.

> Allow MassIndexer to recover gracefully when individual objects won't index
> ---------------------------------------------------------------------------
>
>                 Key: HSEARCH-601
>                 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-601
>             Project: Hibernate Search
>          Issue Type: Improvement
>          Components: massindexer
>    Affects Versions: 3.2.0.Final
>         Environment: Hibernate 3.5.3, SQL Server 2005
>            Reporter: Ben Dotte
>            Assignee: Sanne Grinovero
>             Fix For: 3.3.0.Beta3
>
>
> Some of our sites take hours to index, so it is critical that initial indexing complete even if some of the individual entities fail to index properly. Right now, it appears that the MassIndexer dies if any individual entity throws an exception during indexing, and does not complete indexing for that type.
> I have a workaround for now by overriding EntityConsumerLuceneworkProducer with my own index() method that catches and logs exceptions from docBuilder.createAddWork():
> 	private void index( Object entity, Session session ) throws InterruptedException {
> 		Serializable id = session.getIdentifier( entity );
> 		Class clazz = Hibernate.getClass( entity );
> 		DocumentBuilderIndexedEntity docBuilder = documentBuilders.get( clazz );
> 		TwoWayFieldBridge idBridge = docBuilder.getIdBridge();
> 		String idInString = idBridge.objectToString( id );
> 		//depending on the complexity of the object graph going to be indexed it's possible
> 		//that we hit the database several times during work construction.
> 		try
> 		{
> 			AddLuceneWork addWork = docBuilder.createAddWork( clazz, entity, id, idInString, true );
> 			backend.enqueueAsyncWork( addWork );
> 		}
> 		catch (Exception e)
> 		{
> 			log.error("Error indexing " + clazz + " id " + id, e);
> 		}
> 	}
> This has the added benefit that the object type and id that errored is logged, where that can be tough to track down otherwise.
> That may not be the best solution in general since it could hide exceptions from Hibernate Search itself.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira