[infinispan-issues] [JBoss JIRA] (ISPN-689) Read past EOF caused in Lucene Directory when Lucene flushes but doesn't close the segment

[Horacio Vico (Commented) (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642200#comment-12642200 ] 

Horacio Vico commented on ISPN-689:
-----------------------------------

I am getting a similar exception using Infinispan 'Pagoa' 5.0.1.FINAL.

I have a two node cache, which share a MySQL JDBC CacheStore to store Lucene indexes. This exceptions appear randomly when I restart jboss at the nodes, and the only solution I found after getting this error is to reindex all my entities from scratch.

JAVA Stack trace:

Caused by: java.io.IOException: Read past EOF: Chunk value could not be found for key _0.tis|0|some.package.entity.SomeEntity
	at org.infinispan.lucene.InfinispanIndexInput.setBufferToCurrentChunk(InfinispanIndexInput.java:138)
	at org.infinispan.lucene.InfinispanIndexInput.nextChunk(InfinispanIndexInput.java:131)
	at org.infinispan.lucene.InfinispanIndexInput.readByte(InfinispanIndexInput.java:78)
	at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:71)
	at org.apache.lucene.index.SegmentTermEnum.<init>(SegmentTermEnum.java:50)
	at org.apache.lucene.index.TermInfosReader.<init>(TermInfosReader.java:102)
	at org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:121)
	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:578)
	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:556)
	at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:113)
	at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:29)
	at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:81)
	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:736)
	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
	at org.apache.lucene.index.IndexReader.open(IndexReader.java:428)
	at org.apache.lucene.index.IndexReader.open(IndexReader.java:288)
	at org.hibernate.search.reader.SharingBufferReaderProvider.readerFactory(SharingBufferReaderProvider.java:169)
	at org.hibernate.search.reader.SharingBufferReaderProvider$PerDirectoryLatestReader.<init>(SharingBufferReaderProvider.java:243)
	at org.hibernate.search.reader.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:113)
	... 164 more


                
> Read past EOF caused in Lucene Directory when Lucene flushes but doesn't close the segment
> ------------------------------------------------------------------------------------------
>
>                 Key: ISPN-689
>                 URL: https://issues.jboss.org/browse/ISPN-689
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Lucene Directory
>    Affects Versions: 4.1.0.Final
>            Reporter: Sanne Grinovero
>            Assignee: Sanne Grinovero
>             Fix For: 4.2.0.ALPHA3, 4.2.0.Final
>
>
> While this is not the default access scenario performed by Lucene it's possible in some branches to flush the segment and be able to read it before the close.
> I could reproduce the following stacktrace in high load / huge sized index but not in a unit test; disabling the batching started between .flush() and .close() seems to resolve the issue; also this batch seems totally useless as I couldn't find any chance in performance when disabling it.
> java.io.IOException: Read past EOF: Chunk value could not be found for key _ni.fdt|4|issues
> 	at org.infinispan.lucene.InfinispanIndexInput.setBufferToCurrentChunk(InfinispanIndexInput.java:138)
> 	at org.infinispan.lucene.InfinispanIndexInput.nextChunk(InfinispanIndexInput.java:131)
> 	at org.infinispan.lucene.InfinispanIndexInput.readBytes(InfinispanIndexInput.java:96)
> 	at org.apache.lucene.store.IndexInput.readBytes(IndexInput.java:61)
> 	at org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.java:228)
> 	at org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:184)
> 	at org.apache.lucene.index.IndexWriter.flushDocStores(IndexWriter.java:2342)
> 	at org.apache.lucene.index.IndexWriter.doFlushInternal(IndexWriter.java:4359)
> 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:4264)
> 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:4255)
> 	at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:4133)
> 	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:4206)
> 	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:4179)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira