[hibernate-issues] [Hibernate-JIRA] Commented: (HSEARCH-573) PerDPQueueProcessor forces release of lock even if not held - causes corrupt index
Sanne Grinovero (JIRA)
noreply at atlassian.com
Thu Aug 12 10:18:41 EDT 2010
[ http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=38053#action_38053 ]
Sanne Grinovero commented on HSEARCH-573:
-----------------------------------------
but what's the use case? I expect this to be used offline, not while users are browsing the index: also while it's working many results might be missing as the index is emptied at start and then gradually filled to contain all database matches, but this could take some time, and also it uses many CPU resources: a website might be slow during indexing.
So if you have to reindex periodically, do it on another machine and use rsync or somthing like the Master/Slave directory providers to take snapshots from it.
Ideally you would need the massindexer only for maintenance purposes, or to rebuild the index when you made changes to the model which require a reindex, not "online" on the production system: for this case Hibernate Search will keep the index in sync with the standard event listeners.
> PerDPQueueProcessor forces release of lock even if not held - causes corrupt index
> ----------------------------------------------------------------------------------
>
> Key: HSEARCH-573
> URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-573
> Project: Hibernate Search
> Issue Type: Bug
> Components: engine, massindexer
> Affects Versions: 3.2.0.CR1, 3.2.0.Final, 3.2.1, 3.3.0.Alpha1
> Environment: Hibernate 3.5.0-Final, Oracle 10g
> Reporter: Christian Köberl
> Assignee: Sanne Grinovero
> Fix For: 3.2.2, 3.3.0
>
> Attachments: testHibernateSearchLockFailure.zip
>
>
> Occurs: when an indexed entity is modified while another thread is re-indexing the index for that entity
> Consequences: Hibernate Search calls "workspace.forceLockRelease();" in the catch-block of PerDPQueueProcessor#run - so the lock held by the batch indexer is forcefully released. That means the next entity index operation will write to the index causing the index to be corrupt.
> h2. First Exception - triggers forceLockRelease
> {noformat}
> ERROR| Unexpected error in Lucene Backend: | at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:118)
> org.hibernate.search.SearchException: Unable to open IndexWriter
> at org.hibernate.search.backend.Workspace.getIndexWriter(Workspace.java:159)
> at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:103)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:432)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:678)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:703)
> at java.lang.Thread.run(Thread.java:811)
> Caused by:
> org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: SimpleFSLock at C:\Temp\turntableLuceneIndex\LogEntry\lucene-74da319434c1dd9f133d63245791e1b4-write.lock
> at org.apache.lucene.store.Lock.obtain(Lock.java:85)
> at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1538)
> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1067)
> at org.hibernate.search.backend.Workspace.getIndexWriter(Workspace.java:152)
> ... 7 more
> WARN | going to force release of the IndexWriter lock | at org.hibernate.search.backend.Workspace.forceLockRelease(Workspace.java:221)
> {noformat}
> h2. second Exception - index is destroyed
> {noformat}
> ERROR| Exception occurred org.hibernate.search.SearchException: Unable to add to Lucene index: class com.poi.egh.turntable.vehicle.domain.vehicle.Vehicle#200
> Primary Failure:
> Entity com.poi.egh.turntable.vehicle.domain.vehicle.Vehicle Id 200 Work Type org.hibernate.search.backend.AddLuceneWork
> | at org.hibernate.search.exception.impl.LogErrorHandler.logError(LogErrorHandler.java:83)
> org.hibernate.search.SearchException: Unable to add to Lucene index: class com.poi.egh.turntable.vehicle.domain.vehicle.Vehicle#200
> at org.hibernate.search.backend.impl.lucene.works.AddWorkDelegate.performWork(AddWorkDelegate.java:81)
> at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:106)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:432)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:678)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:703)
> at java.lang.Thread.run(Thread.java:811)
> Caused by:
> java.io.IOException: Cannot overwrite: C:\temp\turntableLuceneIndex\vehicle\_5.fdt
> at org.apache.lucene.store.FSDirectory.initOutput(FSDirectory.java:362)
> at org.apache.lucene.store.SimpleFSDirectory.createOutput(SimpleFSDirectory.java:58)
> at org.apache.lucene.index.FieldsWriter.<init>(FieldsWriter.java:61)
> at org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:66)
> at org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:144)
> at org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:190)
> at org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1466)
> at org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1485)
> at org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1089)
> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:775)
> at org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:750)
> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2454)
> at org.hibernate.search.backend.impl.lucene.works.AddWorkDelegate.performWork(AddWorkDelegate.java:77)
> ... 7 more
> {noformat}
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://opensource.atlassian.com/projects/hibernate/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
More information about the hibernate-issues
mailing list