December 2011 - infinispan-issues

[JBoss JIRA] Created: (ISPN-1362) Hitting open file limit when indexing with Hibernate Search and persistance

by Todd Underwood (JIRA)

Hitting open file limit when indexing with Hibernate Search and persistance --------------------------------------------------------------------------- Key: ISPN-1362 URL: https://issues.jboss.org/browse/ISPN-1362 Project: Infinispan Issue Type: Bug Components: Loaders and Stores, Lucene Directory Affects Versions: 5.0.0.FINAL Environment: OS: Mac OSX 10.6 IDE: IntelliJ IDEA 10 Java: 10.6.0_26 Hibernate Search: 3.4.0.Final Lucene: 3.1.0 jGroups: 2.12.1.3.Final Reporter: Todd Underwood Assignee: Manik Surtani When trying to initalize my index for Hibernate search with persistance I get the following exception after several hours of indexing: [2011-08-29 11:30:53,425] ERROR FileCacheStore.java:317 Hibernate Search: indexwriter-154 ) ISPN000063: Exception while saving bucket Bucket{entries={_4o.fdt|M|cnwk.foreman.model.SoftwareDownload=ImmortalCacheEntry{key=_4o.fdt|M|cnwk.foreman.model.SoftwareDownload, value=ImmortalCacheValue{value=FileMetadata{lastModified=1314642653425, size=32768}}}}, bucketId='1509281792'} java.io.FileNotFoundException: /var/opt/fullTextStore/LuceneIndexesMetadata/1509281792 (Too many open files) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216) at org.infinispan.loaders.file.FileCacheStore$BufferedFileSync.createChannel(FileCacheStore.java:494) at org.infinispan.loaders.file.FileCacheStore$BufferedFileSync.write(FileCacheStore.java:472) at org.infinispan.loaders.file.FileCacheStore.updateBucket(FileCacheStore.java:315) at org.infinispan.loaders.bucket.BucketBasedCacheStore.insertBucket(BucketBasedCacheStore.java:137) at org.infinispan.loaders.bucket.BucketBasedCacheStore.storeLockSafe(BucketBasedCacheStore.java:94) at org.infinispan.loaders.bucket.BucketBasedCacheStore.storeLockSafe(BucketBasedCacheStore.java:49) at org.infinispan.loaders.LockSupportCacheStore.store(LockSupportCacheStore.java:195) at org.infinispan.interceptors.CacheStoreInterceptor.visitPutKeyValueCommand(CacheStoreInterceptor.java:210) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119) at org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java:82) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:133) at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:60) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119) at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:214) at org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:162) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119) at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:114) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:104) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:64) at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:60) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119) at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:77) at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:60) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:274) at org.infinispan.CacheImpl.put(CacheImpl.java:515) at org.infinispan.CacheSupport.put(CacheSupport.java:51) at org.infinispan.lucene.InfinispanIndexOutput.close(InfinispanIndexOutput.java:206) at org.apache.lucene.util.IOUtils.closeSafely(IOUtils.java:80) at org.apache.lucene.index.FieldsWriter.close(FieldsWriter.java:111) at org.apache.lucene.index.FieldsWriter.abort(FieldsWriter.java:121) at org.apache.lucene.index.StoredFieldsWriter.abort(StoredFieldsWriter.java:90) at org.apache.lucene.index.DocFieldProcessor.abort(DocFieldProcessor.java:71) at org.apache.lucene.index.DocumentsWriter.abort(DocumentsWriter.java:421) at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:729) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2042) at org.hibernate.search.backend.impl.lucene.works.AddWorkDelegate.performWork(AddWorkDelegate.java:76) at org.hibernate.search.backend.impl.batchlucene.DirectoryProviderWorkspace.doWorkInSync(DirectoryProviderWorkspace.java:96) at org.hibernate.search.backend.impl.batchlucene.DirectoryProviderWorkspace$AsyncIndexRunnable.run(DirectoryProviderWorkspace.java:144) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) The open file limit on my machine has already been increased to try to fix the issue. This is the configuration used when the exception is thrown: ]<?xml version="1.0" encoding="UTF-8"?> <infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:infinispan:config:4.2 http://www.infinispan.org/schemas/infinispan-config-4.2.xsd" xmlns="urn:infinispan:config:4.2">    <global>  <globalJmxStatistics enabled="true" cacheManagerName="HibernateSearch" allowDuplicateDomains="true"/>  <transport clusterName="HibernateSearch-Infinispan-cluster" distributedSyncTimeout="50000">  </transport>  <shutdown hookBehavior="DONT_REGISTER"/> </global>    <default> <locking lockAcquisitionTimeout="20000" writeSkewCheck="false" concurrencyLevel="500" useLockStriping="false"/> <lazyDeserialization enabled="false"/>  <invocationBatching enabled="true"/>  <clustering mode="replication">  <stateRetrieval timeout="60000" logFlushTimeout="30000" fetchInMemoryState="true" alwaysProvideInMemoryState="true"/>  <sync replTimeout="20000"/> </clustering> <jmxStatistics enabled="true"/> <eviction maxEntries="-1" strategy="NONE"/> <expiration maxIdle="-1"/> </default>             <namedCache name="LuceneIndexesMetadata"> <clustering mode="replication"> <stateRetrieval fetchInMemoryState="true" logFlushTimeout="30000"/> <sync replTimeout="25000"/> </clustering> <loaders preload="true"> <loader class="org.infinispan.loaders.file.FileCacheStore" fetchPersistentState="true"> <properties> <property name="location" value="/var/opt/fullTextStore"/> </properties> </loader> </loaders> </namedCache>    <namedCache name="LuceneIndexesData"> <clustering mode="replication"> <stateRetrieval fetchInMemoryState="true" logFlushTimeout="30000"/> <sync replTimeout="25000"/> </clustering> <loaders> <loader class="org.infinispan.loaders.file.FileCacheStore" fetchPersistentState="true"> <properties> <property name="location" value="/var/opt/fullTextStore"/> </properties> </loader> </loaders> </namedCache>    <namedCache name="LuceneIndexesLocking"> <clustering mode="replication"> <stateRetrieval fetchInMemoryState="true" logFlushTimeout="30000"/> <sync replTimeout="25000"/> </clustering> </namedCache> </infinispan> There are 10160 open files in the cache store when the exception is thrown and a total of 10178 files visible in the cache store. Submitting this so the issue can be tracked after being suggested to do so on Hibernate Search Forums. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

6
22
0 / 0

[JBoss JIRA] Created: (ISPN-445) Ensure that locks time out

by Phil vd (JIRA)

Ensure that locks time out -------------------------- Key: ISPN-445 URL: https://jira.jboss.org/browse/ISPN-445 Project: Infinispan Issue Type: Feature Request Components: Locking and Concurrency Reporter: Phil vd Assignee: Manik Surtani When a lock is acquired, no mechanism ensure that it is reclaimed later. So if for whatever reason the thread owning the lock is stuck in an endless loop or waiting for some resource, the entry is locked. This kind of situation could happen in cache stores trying to contact a remote storage. The risk is quite high that other entries may be locked as well if you use lock stripping (default : TRUE) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

5
5
0 / 0

[JBoss JIRA] Created: (ISPN-627) Provision to get Cache from CacheManager, without starting the Cache

by Girish Adat (JIRA)

Provision to get Cache from CacheManager, without starting the Cache -------------------------------------------------------------------- Key: ISPN-627 URL: https://jira.jboss.org/browse/ISPN-627 Project: Infinispan Issue Type: Feature Request Components: Core API Affects Versions: 4.0.0.Final Reporter: Girish Adat Assignee: Manik Surtani Provision to getCache() from CacheManager, without starting the cache. This is for achieving scenarios like adding cache-listeners before starting the cache for first time. Adding a method getCache(String cacheName, boolean start) will work for the requirement. Also, any change in cache level configuration can be made programmatically by the client application, if this method is implemented. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

5
15
0 / 0

[JBoss JIRA] (ISPN-1450) Improve performance of tree cache writes in nodes with a lot of data

by Galder Zamarreño (Created) (JIRA)

Improve performance of tree cache writes in nodes with a lot of data -------------------------------------------------------------------- Key: ISPN-1450 URL: https://issues.jboss.org/browse/ISPN-1450 Project: Infinispan Issue Type: Bug Components: Tree API Reporter: Galder Zamarreño Assignee: Galder Zamarreño Fix For: 5.2.0.FINAL Look into speeding up tree cache performance when it comes to storing large amount of key/value pairs in a single node. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

3
2
0 / 0

[JBoss JIRA] Created: (ISPN-984) Add ability to retrieve cache names remotely in Hot Rod

by Galder Zamarreño (JIRA)

Add ability to retrieve cache names remotely in Hot Rod ------------------------------------------------------- Key: ISPN-984 URL: https://issues.jboss.org/browse/ISPN-984 Project: Infinispan Issue Type: Feature Request Components: Cache Server Reporter: Galder Zamarreño Assignee: Galder Zamarreño Fix For: 5.0.0.Final A new operation would be needed to retrieve all cache names that are configured and running. A new version of the protocol would need to be set up to deal with the new operation. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

4
6
0 / 0

[JBoss JIRA] Created: (ISPN-506) Infinispan should have a notion of comparable dataversion for handling external db updates

by Max Rydahl Andersen (JIRA)

Infinispan should have a notion of comparable dataversion for handling external db updates ------------------------------------------------------------------------------------------ Key: ISPN-506 URL: https://jira.jboss.org/browse/ISPN-506 Project: Infinispan Issue Type: Feature Request Reporter: Max Rydahl Andersen Assignee: Manik Surtani As discussed in hotel bar -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

6
18
0 / 0

[JBoss JIRA] Created: (ISPN-755) Provide API documentation for scala components within the distribution

by Richard Achmatowicz (JIRA)

Provide API documentation for scala components within the distribution ---------------------------------------------------------------------- Key: ISPN-755 URL: https://jira.jboss.org/browse/ISPN-755 Project: Infinispan Issue Type: Feature Request Affects Versions: 4.1.0.Final Reporter: Richard Achmatowicz Assignee: Manik Surtani Infinispan makes use of the Scala language to implement server modules for client-server communication. At present, these modules do not have any API documentation, which means that application developers must browse through the source code in order to find out which classes are available and what their interfaces are. Scala has the ability to produce scaladoc documentation, but this is not compatible with javadoc. It appears that there is no method at present to incorporate scala API information into existing javadoc documentation. The task of this issue is to investigate and implement a viable way to incorporate API information for the scala classes into the Infinispan distribution. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

7
31
0 / 0

[JBoss JIRA] Created: (ISPN-1001) Should handle 3-way (or more) merges

by Manik Surtani (JIRA)

Should handle 3-way (or more) merges ------------------------------------ Key: ISPN-1001 URL: https://issues.jboss.org/browse/ISPN-1001 Project: Infinispan Issue Type: Feature Request Components: Distributed Cache Affects Versions: 4.2.0.Final Reporter: Manik Surtani Assignee: Manik Surtani Fix For: 5.0.0.BETA2, 5.0.0.FINAL The current fix for handling merge views (ISPN-923) only handles to 2-way merges reliably. A split of a network into even more partitions could result in data loss. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

5
17
0 / 0

[JBoss JIRA] Created: (ISPN-1042) Ensure proper remote distributed task interruption/cancellation

by Vladimir Blagojevic (JIRA)

Ensure proper remote distributed task interruption/cancellation ---------------------------------------------------------------- Key: ISPN-1042 URL: https://issues.jboss.org/browse/ISPN-1042 Project: Infinispan Issue Type: Task Components: Distributed Cache Affects Versions: 5.0.0.BETA1 Reporter: Vladimir Blagojevic Assignee: Vladimir Blagojevic Fix For: 5.0.0.CR1 We need to implement remote task interruption and cancellation for distributed executors and map reduce. Also, implement a proper shutdown of DistributedExecutorService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

4
9
0 / 0

[JBoss JIRA] Created: (ISPN-1359) SetExternalizer Cannot Properly Handle TreeSets Constructed With Comparator

by Ryan Scharer (JIRA)

SetExternalizer Cannot Properly Handle TreeSets Constructed With Comparator --------------------------------------------------------------------------- Key: ISPN-1359 URL: https://issues.jboss.org/browse/ISPN-1359 Project: Infinispan Issue Type: Bug Components: Marshalling Affects Versions: 5.0.0.FINAL Environment: OSX 10.7.1, System JDK Reporter: Ryan Scharer Assignee: Manik Surtani TreeSets can be used to hold objects that don't implement Comparable, so long as you provide a Comparator during construction. The current implementation of SetExternalizer doesn't support this pattern... trying to deserialize a TreeSet that holds objects that don't implement Comparable results in a ClassCastException. This seriously reduces the flexibility of TreeSet. JBoss Cache doesn't have this limitation, presumably because it delegates to native Java serialization. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

12 years, 1 month

4
7
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues December 2011