[JBoss JIRA] (ISPN-2020) MarshalledValues to not compact eagerly
by Manik Surtani (JIRA)
Manik Surtani created ISPN-2020:
-----------------------------------
Summary: MarshalledValues to not compact eagerly
Key: ISPN-2020
URL: https://issues.jboss.org/browse/ISPN-2020
Project: Infinispan
Issue Type: Feature Request
Components: Marshalling
Affects Versions: 5.1.4.FINAL
Reporter: Manik Surtani
Assignee: Galder Zamarreño
Fix For: 5.1.x, 5.2.0.ALPHA1, 5.2.0.FINAL
Marshalled values currently get compacted aggressively, limiting usability as a form of class loader isolation. E.g.,
Node 1: put(k, v) results in v being wrapped in a MarshalledValue, the instance field being set, the raw field being generated when serialization is needed, and as the put call returns, the raw field is un-set. This is fine for the most part, except when state transfer kicks in, and the entry needs to be transferred elsewhere. The instance may not be serialized since an application specific class loader may not be available at this time.
This proposal is to add the following:
1. A new configuration option, {{eagerCompacting}}, defaulting to {{true}} (current behaviour).
2. If set to false, the raw representation is kept around, and can be reused, effectively working around the class loader problem.
3. The drawback here would be the additional memory needed to store the second reference on the originator, but the benefit of not having to serialize as often should offset this.
Compacting then can still happen on the originator by manually triggering the {{cache#compact()}} API.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] (ISPN-1439) Async store needs redesigning to support injected executors
by Galder Zamarreño (Created) (JIRA)
Async store needs redesigning to support injected executors
-----------------------------------------------------------
Key: ISPN-1439
URL: https://issues.jboss.org/browse/ISPN-1439
Project: Infinispan
Issue Type: Enhancement
Components: Configuration, Loaders and Stores
Reporter: Galder Zamarreño
Assignee: Sanne Grinovero
Fix For: 5.2.0.FINAL
The CoalescedAsyncStore design is pretty complex and is designed around AsyncStore being able to control the lifecycle of the async store coordinator and processor executors.
A re-design of the async store is needed to enable central management of these executors, in such way that the NamedExecutorFactory can start with the right parameters (take in account that multiple caches could be configured with an async store), and the shutdown procedure can be correctly executed by the cache manager.
The async store's coalesced logic makes this a fairly complex task, particularly since the introduction of Sanne's changes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] Created: (ISPN-924) Support Atomic arithmetic operations in the API
by Erik Salter (JIRA)
Support Atomic arithmetic operations in the API
-----------------------------------------------
Key: ISPN-924
URL: https://issues.jboss.org/browse/ISPN-924
Project: Infinispan
Issue Type: Feature Request
Reporter: Erik Salter
Assignee: Manik Surtani
It would be useful if Infinispan supported cluster-wide atomic operations for java.lang.Number types in the API -- similar to the AtomicX API in Java.
Right now, to cobble this functionality, we'd need to do something like:
- Start a tx
- Single-lock a key
- Get and increment
- Commit
- And, of course, handle the cache exceptions =)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] Created: (ISPN-1208) Use Hibernate Validator to validate entries stored in Infinispan
by Manik Surtani (JIRA)
Use Hibernate Validator to validate entries stored in Infinispan
----------------------------------------------------------------
Key: ISPN-1208
URL: https://issues.jboss.org/browse/ISPN-1208
Project: Infinispan
Issue Type: Feature Request
Components: Core API
Affects Versions: 5.0.0.FINAL
Reporter: Manik Surtani
Assignee: Manik Surtani
Fix For: 5.1.0.BETA1, 5.1.0.Final
Should be very trivial to implement as an additional interceptor, where values can be annotated with Hibernate Validator and the interceptor can validate the entries passed in.
Should be optional, and packaged as a separate module so that we don't pull in additional dependencies into core.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] Created: (ISPN-1301) Check hashKeyMask of FIleCacheStore
by Robert Stupp (JIRA)
Check hashKeyMask of FIleCacheStore
-----------------------------------
Key: ISPN-1301
URL: https://issues.jboss.org/browse/ISPN-1301
Project: Infinispan
Issue Type: Enhancement
Components: Configuration, Loaders and Stores
Affects Versions: 5.0.0.CR8
Reporter: Robert Stupp
Assignee: Manik Surtani
Related to ISPN-1300:
If the configured "hashKeyMask" value is changed for an existing cache store, the whole cache store becomes unusable.
Parts of the cache store configuration should be persisted in the file cache store and checked upon startup.
If the existing (persistent) configuration is different from what the user has set in the configuration XML (or programatically), two things are possible:
1. print a WARNING that the cache store uses the persistent configuration
2. like 1, but perform a re-hash of the file cache store, if a certain configuration option has been set
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] Created: (ISPN-1362) Hitting open file limit when indexing with Hibernate Search and persistance
by Todd Underwood (JIRA)
Hitting open file limit when indexing with Hibernate Search and persistance
---------------------------------------------------------------------------
Key: ISPN-1362
URL: https://issues.jboss.org/browse/ISPN-1362
Project: Infinispan
Issue Type: Bug
Components: Loaders and Stores, Lucene Directory
Affects Versions: 5.0.0.FINAL
Environment: OS: Mac OSX 10.6
IDE: IntelliJ IDEA 10
Java: 10.6.0_26
Hibernate Search: 3.4.0.Final
Lucene: 3.1.0
jGroups: 2.12.1.3.Final
Reporter: Todd Underwood
Assignee: Manik Surtani
When trying to initalize my index for Hibernate search with persistance I get the following exception after several hours of indexing:
[2011-08-29 11:30:53,425] ERROR FileCacheStore.java:317 Hibernate Search: indexwriter-154 ) ISPN000063: Exception while saving bucket Bucket{entries={_4o.fdt|M|cnwk.foreman.model.SoftwareDownload=ImmortalCacheEntry{key=_4o.fdt|M|cnwk.foreman.model.SoftwareDownload, value=ImmortalCacheValue{value=FileMetadata{lastModified=1314642653425, size=32768}}}}, bucketId='1509281792'}
java.io.FileNotFoundException: /var/opt/fullTextStore/LuceneIndexesMetadata/1509281792 (Too many open files)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
at org.infinispan.loaders.file.FileCacheStore$BufferedFileSync.createChannel(FileCacheStore.java:494)
at org.infinispan.loaders.file.FileCacheStore$BufferedFileSync.write(FileCacheStore.java:472)
at org.infinispan.loaders.file.FileCacheStore.updateBucket(FileCacheStore.java:315)
at org.infinispan.loaders.bucket.BucketBasedCacheStore.insertBucket(BucketBasedCacheStore.java:137)
at org.infinispan.loaders.bucket.BucketBasedCacheStore.storeLockSafe(BucketBasedCacheStore.java:94)
at org.infinispan.loaders.bucket.BucketBasedCacheStore.storeLockSafe(BucketBasedCacheStore.java:49)
at org.infinispan.loaders.LockSupportCacheStore.store(LockSupportCacheStore.java:195)
at org.infinispan.interceptors.CacheStoreInterceptor.visitPutKeyValueCommand(CacheStoreInterceptor.java:210)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119)
at org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java:82)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:133)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:60)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119)
at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:214)
at org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:162)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119)
at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:114)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:104)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:64)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:60)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:119)
at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:77)
at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:60)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:274)
at org.infinispan.CacheImpl.put(CacheImpl.java:515)
at org.infinispan.CacheSupport.put(CacheSupport.java:51)
at org.infinispan.lucene.InfinispanIndexOutput.close(InfinispanIndexOutput.java:206)
at org.apache.lucene.util.IOUtils.closeSafely(IOUtils.java:80)
at org.apache.lucene.index.FieldsWriter.close(FieldsWriter.java:111)
at org.apache.lucene.index.FieldsWriter.abort(FieldsWriter.java:121)
at org.apache.lucene.index.StoredFieldsWriter.abort(StoredFieldsWriter.java:90)
at org.apache.lucene.index.DocFieldProcessor.abort(DocFieldProcessor.java:71)
at org.apache.lucene.index.DocumentsWriter.abort(DocumentsWriter.java:421)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:729)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2042)
at org.hibernate.search.backend.impl.lucene.works.AddWorkDelegate.performWork(AddWorkDelegate.java:76)
at org.hibernate.search.backend.impl.batchlucene.DirectoryProviderWorkspace.doWorkInSync(DirectoryProviderWorkspace.java:96)
at org.hibernate.search.backend.impl.batchlucene.DirectoryProviderWorkspace$AsyncIndexRunnable.run(DirectoryProviderWorkspace.java:144)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
The open file limit on my machine has already been increased to try to fix the issue.
This is the configuration used when the exception is thrown:
]<?xml version="1.0" encoding="UTF-8"?>
<infinispan
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:4.2 http://www.infinispan.org/schemas/infinispan-config-4.2.xsd"
xmlns="urn:infinispan:config:4.2">
<!-- *************************** -->
<!-- System-wide global settings -->
<!-- *************************** -->
<global>
<!-- Duplicate domains are allowed so that multiple deployments with default configuration
of Hibernate Search applications work - if possible it would be better to use JNDI to share
the CacheManager across applications -->
<globalJmxStatistics
enabled="true"
cacheManagerName="HibernateSearch"
allowDuplicateDomains="true"/>
<!-- If the transport is omitted, there is no way to create distributed or clustered
caches. There is no added cost to defining a transport but not creating a cache that uses one,
since the transport is created and initialized lazily. -->
<transport
clusterName="HibernateSearch-Infinispan-cluster"
distributedSyncTimeout="50000">
<!-- Note that the JGroups transport uses sensible defaults if no configuration
property is defined. See the JGroupsTransport javadocs for more flags -->
</transport>
<!-- Used to register JVM shutdown hooks. hookBehavior: DEFAULT, REGISTER, DONT_REGISTER.
Hibernate Search takes care to stop the CacheManager so registering is not needed -->
<shutdown
hookBehavior="DONT_REGISTER"/>
</global>
<!-- *************************** -->
<!-- Default "template" settings -->
<!-- *************************** -->
<default>
<locking
lockAcquisitionTimeout="20000"
writeSkewCheck="false"
concurrencyLevel="500"
useLockStriping="false"/>
<lazyDeserialization
enabled="false"/>
<!-- Invocation batching is required for use with the Lucene Directory -->
<invocationBatching
enabled="true"/>
<!-- This element specifies that the cache is clustered. modes supported: distribution
(d), replication (r) or invalidation (i). Don't use invalidation to store Lucene indexes (as
with Hibernate Search DirectoryProvider). Replication is recommended for best performance of
Lucene indexes, but make sure you have enough memory to store the index in your heap.
Also distribution scales much better than replication on high number of nodes in the cluster. -->
<clustering
mode="replication">
<!-- Prefer loading all data at startup than later -->
<stateRetrieval
timeout="60000"
logFlushTimeout="30000"
fetchInMemoryState="true"
alwaysProvideInMemoryState="true"/>
<!-- Network calls are synchronous by default -->
<sync
replTimeout="20000"/>
</clustering>
<jmxStatistics
enabled="true"/>
<eviction
maxEntries="-1"
strategy="NONE"/>
<expiration
maxIdle="-1"/>
</default>
<!-- ******************************************************************************* -->
<!-- Individually configured "named" caches. -->
<!-- -->
<!-- While default configuration happens to be fine with similar settings across the -->
<!-- three caches, they should generally be different in a production environment. -->
<!-- -->
<!-- Current settings could easily lead to OutOfMemory exception as a CacheStore -->
<!-- should be enabled, and maybe distribution is desired. -->
<!-- ******************************************************************************* -->
<!-- *************************************** -->
<!-- Cache to store Lucene's file metadata -->
<!-- *************************************** -->
<namedCache name="LuceneIndexesMetadata">
<clustering mode="replication">
<stateRetrieval
fetchInMemoryState="true"
logFlushTimeout="30000"/>
<sync replTimeout="25000"/>
</clustering>
<loaders preload="true">
<loader class="org.infinispan.loaders.file.FileCacheStore" fetchPersistentState="true">
<properties>
<property name="location" value="/var/opt/fullTextStore"/>
</properties>
</loader>
</loaders>
</namedCache>
<!-- **************************** -->
<!-- Cache to store Lucene data -->
<!-- **************************** -->
<namedCache name="LuceneIndexesData">
<clustering mode="replication">
<stateRetrieval
fetchInMemoryState="true"
logFlushTimeout="30000"/>
<sync
replTimeout="25000"/>
</clustering>
<loaders>
<loader class="org.infinispan.loaders.file.FileCacheStore" fetchPersistentState="true">
<properties>
<property name="location" value="/var/opt/fullTextStore"/>
</properties>
</loader>
</loaders>
</namedCache>
<!-- ***************************** -->
<!-- Cache to store Lucene locks -->
<!-- ***************************** -->
<namedCache
name="LuceneIndexesLocking">
<clustering
mode="replication">
<stateRetrieval
fetchInMemoryState="true"
logFlushTimeout="30000"/>
<sync
replTimeout="25000"/>
</clustering>
</namedCache>
</infinispan>
There are 10160 open files in the cache store when the exception is thrown and a total of 10178 files visible in the cache store.
Submitting this so the issue can be tracked after being suggested to do so on Hibernate Search Forums.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months
[JBoss JIRA] Created: (ISPN-445) Ensure that locks time out
by Phil vd (JIRA)
Ensure that locks time out
--------------------------
Key: ISPN-445
URL: https://jira.jboss.org/browse/ISPN-445
Project: Infinispan
Issue Type: Feature Request
Components: Locking and Concurrency
Reporter: Phil vd
Assignee: Manik Surtani
When a lock is acquired, no mechanism ensure that it is reclaimed later.
So if for whatever reason the thread owning the lock is stuck in an endless loop or waiting for some resource, the entry is locked. This kind of situation could happen in cache stores trying to contact a remote storage.
The risk is quite high that other entries may be locked as well if you use lock stripping (default : TRUE)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 2 months