[hibernate-issues] [Hibernate-JIRA] Resolved: (HSEARCH-573) PerDPQueueProcessor forces release of lock even if not held - causes corrupt index

[Emmanuel Bernard (JIRA)]

     [ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Emmanuel Bernard resolved HSEARCH-573.
--------------------------------------

         Assignee: Emmanuel Bernard  (was: Sanne Grinovero)
       Resolution: Fixed
    Fix Version/s:     (was: 3.3.0)
                   3.3.0.CR1

I've fixed the corruption issue.
The idea of being able to run MassIndexer in parallel is a whole different feature, I will open a dedicated jira for it.

> PerDPQueueProcessor forces release of lock even if not held - causes corrupt index
> ----------------------------------------------------------------------------------
>
>                 Key: HSEARCH-573
>                 URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-573
>             Project: Hibernate Search
>          Issue Type: Bug
>          Components: engine, massindexer
>    Affects Versions: 3.2.0.CR1, 3.2.0.Final, 3.2.1, 3.3.0.Alpha1
>         Environment: Hibernate 3.5.0-Final, Oracle 10g
>            Reporter: Christian Köberl
>            Assignee: Emmanuel Bernard
>             Fix For: 3.2.2, 3.3.0.CR1
>
>         Attachments: testHibernateSearchLockFailure.zip
>
>
> Occurs: when an indexed entity is modified while another thread is re-indexing the index for that entity
> Consequences: Hibernate Search calls "workspace.forceLockRelease();" in the catch-block of PerDPQueueProcessor#run - so the lock held by the batch indexer is forcefully released. That means the next entity index operation will write to the index causing the index to be corrupt.
> h2. First Exception - triggers forceLockRelease
> {noformat} 
> ERROR| Unexpected error in Lucene Backend:                                                                                      | at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:118)
> org.hibernate.search.SearchException: Unable to open IndexWriter
> 	at org.hibernate.search.backend.Workspace.getIndexWriter(Workspace.java:159)
> 	at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:103)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:432)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:678)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:703)
> 	at java.lang.Thread.run(Thread.java:811)
> Caused by: 
> org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock at C:\Temp\turntableLuceneIndex\LogEntry\lucene-74da319434c1dd9f133d63245791e1b4-write.lock
> 	at org.apache.lucene.store.Lock.obtain(Lock.java:85)
> 	at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1538)
> 	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1067)
> 	at org.hibernate.search.backend.Workspace.getIndexWriter(Workspace.java:152)
> 	... 7 more
> WARN | going to force release of the IndexWriter lock                                                                           | at org.hibernate.search.backend.Workspace.forceLockRelease(Workspace.java:221)
> {noformat} 
> h2. second Exception - index is destroyed
> {noformat} 
> ERROR| Exception occurred org.hibernate.search.SearchException: Unable to add to Lucene index: class com.poi.egh.turntable.vehicle.domain.vehicle.Vehicle#200
> Primary Failure:
> 	Entity com.poi.egh.turntable.vehicle.domain.vehicle.Vehicle  Id 200  Work Type  org.hibernate.search.backend.AddLuceneWork
>  | at org.hibernate.search.exception.impl.LogErrorHandler.logError(LogErrorHandler.java:83)
> org.hibernate.search.SearchException: Unable to add to Lucene index: class com.poi.egh.turntable.vehicle.domain.vehicle.Vehicle#200
> 	at org.hibernate.search.backend.impl.lucene.works.AddWorkDelegate.performWork(AddWorkDelegate.java:81)
> 	at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:106)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:432)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:678)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:703)
> 	at java.lang.Thread.run(Thread.java:811)
> Caused by: 
> java.io.IOException: Cannot overwrite: C:\temp\turntableLuceneIndex\vehicle\_5.fdt
> 	at org.apache.lucene.store.FSDirectory.initOutput(FSDirectory.java:362)
> 	at org.apache.lucene.store.SimpleFSDirectory.createOutput(SimpleFSDirectory.java:58)
> 	at org.apache.lucene.index.FieldsWriter.<init>(FieldsWriter.java:61)
> 	at org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:66)
> 	at org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:144)
> 	at org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:190)
> 	at org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1466)
> 	at org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1485)
> 	at org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1089)
> 	at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:775)
> 	at org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:750)
> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2454)
> 	at org.hibernate.search.backend.impl.lucene.works.AddWorkDelegate.performWork(AddWorkDelegate.java:77)
> 	... 7 more
> {noformat}

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira