Hello,
When using the Lucene directory in production we have encountered many instances of files being removed incorrectly.
After much investigation, we found issues and logged:
After a long period of stability, we've still found rare problems, relating to when applications are shutdown. I don't need to go into the details here.
We've now decided rather than trying to fix the problems we would enhance the DistributedSegmentReadLocker to prevent unwarranted file deletes. The mechanism is quite simple, in that when the directory really wants a file deleted we would add an additional marker to the cache and then proceed with a real delete only if this marker is present. You can see the changes here:
Would appreciate feedback on this idea and whether we should create a feature request along with pull request.
Thanks
Anuj Shah