July 2012 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-1808) Make InfiniSpan passivation faster

by Johann Burkard (JIRA)

Johann Burkard created ISPN-1808: ------------------------------------ Summary: Make InfiniSpan passivation faster Key: ISPN-1808 URL: https://issues.jboss.org/browse/ISPN-1808 Project: Infinispan Issue Type: Enhancement Components: Loaders and Stores, Marshalling Affects Versions: 5.1.0.FINAL, 5.1.0.CR4 Reporter: Johann Burkard Assignee: Galder Zamarreño While InfiniSpan is very fast in-memory, once passivation is turned on, things get slow for me. On my i5-2400 with a Crucial C300 SSD, it takes about 11 ms to passivate a 2964 B value. This comes down to a write speed of 250 to 300 KB/s. The disk can write up to 180 MB/s so it's not the bottleneck. Clearly, passivation should be faster than that. Profiling with YourKit shows most of the hot spots are centered around {{FileCacheStore}}, the JBoss Marshalling library and various bucket and {{ImmortalCacheEntry}} methods. Locking also seems to play a role. Here's a screenshot: !http://i.imgur.com/LzkPw.png! To make profiling easier for you, I've taken my test case and turned it into a small Maven project. Simply running {{mvn install}} should run the test, ready for profiling. Of course, it could also be that my configuration is suboptimal. However, in my experiments, many configurations were unusable (hitting the file handle limit), which is not a good thing. The project is available at http://johannburkard.de/resources/Johann/infinispan-performance.zip I've also found one or two other things I think are more costly than they would need to be In {{FileCacheStore}} {code} byte[] buffer = new byte[streamBufferSize]; ... fileInStream = new FileInputStream(file); ... bis = new BufferedInputStream(fileInStream); ... bytesRead = bis.read(buffer, 0, streamBufferSize); {code} wrapping the {{FileInputStream}} in a {{BufferedInputStream}} is unnecessary because you already have a buffer in {{byte[] buffer}}. I think I have seen this before in InfiniSpan, so you might want to check for occurrences of {{BufferedInputStream}} or {{BufferedOutputStream}}. Another one {code} if (bytes.length == 0) { // Short circuit if (f.exists()) f.delete(); return; } {code} {{f.exists()}} is essentially a syscall. Just calling {{f.delete()}} is enough. {code} if (!root.exists()) { if (!root.mkdirs()) { log.problemsCreatingDirectory(root); } } if (!root.exists()) { throw new ConfigurationException("Directory " + root.getAbsolutePath() + " does not exist and cannot be created!"); } {code} This could also be shortened to {code} if (!root.mkdirs()) { throw new ConfigurationException("Directory " + root.getAbsolutePath() + " does not exist and cannot be created!"); } {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

5
10
0 / 0

[JBoss JIRA] Created: (ISPN-1001) Should handle 3-way (or more) merges

by Manik Surtani (JIRA)

Should handle 3-way (or more) merges ------------------------------------ Key: ISPN-1001 URL: https://issues.jboss.org/browse/ISPN-1001 Project: Infinispan Issue Type: Feature Request Components: Distributed Cache Affects Versions: 4.2.0.Final Reporter: Manik Surtani Assignee: Manik Surtani Fix For: 5.0.0.BETA2, 5.0.0.FINAL The current fix for handling merge views (ISPN-923) only handles to 2-way merges reliably. A split of a network into even more partitions could result in data loss. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

5
17
0 / 0

[JBoss JIRA] (ISPN-1989) Improve AtomicMap performance

by Galder Zamarreño (JIRA)

Galder Zamarreño created ISPN-1989: -------------------------------------- Summary: Improve AtomicMap performance Key: ISPN-1989 URL: https://issues.jboss.org/browse/ISPN-1989 Project: Infinispan Issue Type: Enhancement Reporter: Galder Zamarreño Assignee: Manik Surtani Fix For: 5.2.0.FINAL Improve performance of AtomicMaps. Ankur has provided a test and has some figures. Use the test to look into performance of AtomicMaps. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

4
3
0 / 0

[JBoss JIRA] Created: (ISPN-1042) Ensure proper remote distributed task interruption/cancellation

by Vladimir Blagojevic (JIRA)

Ensure proper remote distributed task interruption/cancellation ---------------------------------------------------------------- Key: ISPN-1042 URL: https://issues.jboss.org/browse/ISPN-1042 Project: Infinispan Issue Type: Task Components: Distributed Cache Affects Versions: 5.0.0.BETA1 Reporter: Vladimir Blagojevic Assignee: Vladimir Blagojevic Fix For: 5.0.0.CR1 We need to implement remote task interruption and cancellation for distributed executors and map reduce. Also, implement a proper shutdown of DistributedExecutorService. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

4
9
0 / 0

[JBoss JIRA] Created: (ISPN-1359) SetExternalizer Cannot Properly Handle TreeSets Constructed With Comparator

by Ryan Scharer (JIRA)

SetExternalizer Cannot Properly Handle TreeSets Constructed With Comparator --------------------------------------------------------------------------- Key: ISPN-1359 URL: https://issues.jboss.org/browse/ISPN-1359 Project: Infinispan Issue Type: Bug Components: Marshalling Affects Versions: 5.0.0.FINAL Environment: OSX 10.7.1, System JDK Reporter: Ryan Scharer Assignee: Manik Surtani TreeSets can be used to hold objects that don't implement Comparable, so long as you provide a Comparator during construction. The current implementation of SetExternalizer doesn't support this pattern... trying to deserialize a TreeSet that holds objects that don't implement Comparable results in a ClassCastException. This seriously reduces the flexibility of TreeSet. JBoss Cache doesn't have this limitation, presumably because it delegates to native Java serialization. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

4
7
0 / 0

[JBoss JIRA] Created: (ISPN-328) Async store to support frequency based flushing

by Galder Zamarreno (JIRA)

Async store to support frequency based flushing ----------------------------------------------- Key: ISPN-328 URL: https://jira.jboss.org/jira/browse/ISPN-328 Project: Infinispan Issue Type: Feature Request Components: Loaders and Stores Reporter: Galder Zamarreno Assignee: Galder Zamarreno Fix For: 4.1.0.BETA1 Implement frequency based flushing in the async store. This is also called, Scheduled Write-Behing Strategy. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

5
11
0 / 0

[JBoss JIRA] Created: (ISPN-604) Re-design CacheStore transactions

by Mircea Markus (JIRA)

Re-design CacheStore transactions ---------------------------------- Key: ISPN-604 URL: https://jira.jboss.org/browse/ISPN-604 Project: Infinispan Issue Type: Feature Request Components: Loaders and Stores, Transactions Affects Versions: 4.1.0.CR2, 4.0.0.Final Reporter: Mircea Markus Assignee: Mircea Markus Fix For: 5.0.0.Final Current(4.1.x) transaction implementation in CacheStores is brocken in several ways: 1st problem. - AbstractCacheStore.prepare: public void prepare(List<? extends Modification> mods, GlobalTransaction tx, boolean isOnePhase) throws CacheLoaderException { if (isOnePhase) { applyModifications(mods); } else { transactions.put(tx, mods); } } If this is 1PC we apply the modifications in the prepare phase - we should do it in the commit phase (as JTA does it). 2nd problem. This currently exhibits during commit/rollback with JdbcXyzCacheStore, but it is rather a more general cache store issue. When using a TransactionManager, during TM.commit AbstractCacheStore.commit is being called internally which tries to apply all the modifications that happened during that transaction. Within the scope of AbstractCacheStore.commit, JdbcStore obtains a connection from a DataSource and tries to write the modifications on that connection. Now if the DataSource is managed (e.g. by an A.S.) on the DS.getConnection call the A.S. would try to enlist the connection with the ongoing transaction by calling Transaction.enlistResource(XAResource xaRes) [1] This method fails with an IllegalStateException, because the transaction's status is preparing (see javax.transaction.Transaction.enlistResource). Suggested fix: - the modifications should be registered to the transaction as they happen(vs. during prepare/commit as it happens now) - this requires API changes in CacheStore, e.g. void store(InternalCacheEntry entry) should become void store(InternalCacheEntry entry, GlobalTransaction gtx) (gtx would be null if this is not a transactional call). [1] This behavior is specified by the JDBC 2.0 Standard Extension API, chapter 7 - distributed transaction -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

4
9
0 / 0

[JBoss JIRA] (ISPN-2033) JdbcBinaryCacheStore.purgeInternal() - releaseConnection() should be called in finally block.

by Jim Dunkerton (JIRA)

Jim Dunkerton created ISPN-2033: ----------------------------------- Summary: JdbcBinaryCacheStore.purgeInternal() - releaseConnection() should be called in finally block. Key: ISPN-2033 URL: https://issues.jboss.org/browse/ISPN-2033 Project: Infinispan Issue Type: Bug Components: Loaders and Stores Affects Versions: 5.1.2.FINAL Environment: JBoss 7.1.1 Final Reporter: Jim Dunkerton Assignee: Manik Surtani A potential memory leak of pooled DB connections is present in JdbcCacheBinaryStore. Observe the catch and finally blocks of the purgeInternal() method: catch (SQLException ex) { //if something happens make sure buckets locks are being release releaseLocks(expiredBuckets); connectionFactory.releaseConnection(conn); log.failedClearingJdbcCacheStore(ex); throw new CacheLoaderException("Failed clearing JdbcBinaryCacheStore", ex); } finally { JdbcUtil.safeClose(ps); JdbcUtil.safeClose(rs); } The pooled DB connection is not released unless an SQLException has been thrown. Contrast with JdbcStringBasedCacheStore.purgeInternal(): } catch (SQLException ex) { log.failedClearingJdbcCacheStore(ex); throw new CacheLoaderException("Failed clearing string based JDBC store", ex); } finally { JdbcUtil.safeClose(ps); connectionFactory.releaseConnection(conn); } When using a JdbcMixedCacheStore, I noticed a DB connection leak. It appeared that for every two connections being acquired, only one was being released. I believe it is down to the fact that the JdbcBinaryCacheStore inside the JdbcMixedCacheStore is not releasing its connections properly (i.e. in its finally block) whereas the JdbcStringBasedCacheStore is. I guess a workaround is to use only a JdbcStringBasedCacheStore, not a JdbcMixedCacheStore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

5
10
0 / 0

[JBoss JIRA] Created: (ISPN-503) collate keys in replication queue

by Mircea Markus (JIRA)

collate keys in replication queue ---------------------------------- Key: ISPN-503 URL: https://jira.jboss.org/browse/ISPN-503 Project: Infinispan Issue Type: Feature Request Components: Core API Affects Versions: 4.1.0.BETA2 Reporter: Mircea Markus Assignee: Manik Surtani At the moment the replication queue is replicating all the keys. What it can do is to collate them, similar to the way cache stores are collating writes. E.g. 1. put(k,v1) 2. put(k,v2). This can be collated in put(k,v2), as put(k,v1) is redundant. This would optimize async replication in a way that cannot be handled by jgroups. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

4
8
0 / 0

[JBoss JIRA] Created: (ISPN-159) Extended coalesced logic to other parts

by Galder Zamarreno (JIRA)

Extended coalesced logic to other parts --------------------------------------- Key: ISPN-159 URL: https://jira.jboss.org/jira/browse/ISPN-159 Project: Infinispan Issue Type: Feature Request Components: RPC, Transactions Reporter: Galder Zamarreno Assignee: Galder Zamarreno Fix For: 4.0.0.CR1 Reuse same coalesced async cache store logic to: - Reduce size of replicable commands where N modifications are sent so that only latest state is sent. - Manage the async replication queue. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira

11 years, 6 months

3
9
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues July 2012