[infinispan-dev] Faster LRU

Thu Jul 7 06:21:50 EDT 2011

I think we leave the old LRU as LRU_OLD and mark it as deprecated.  

Vladimir, does this apply to LIRS as well?

On 6 Jul 2011, at 21:08, Vladimir Blagojevic wrote:

> No :-) We can leave it, it all depends on how many more CR cycles we have before Final release to field test new LRU!
> 
> On 11-07-06 2:46 PM, Sanne Grinovero wrote:
>> 
>> Awesome!
>> Are you really sure about eradicating the old implementation?
>> 
>> Cheers,
>> Sanne
>> 
>> On 6 Jul 2011 19:39, "Vladimir Blagojevic" <vblagoje at redhat.com> wrote:
>> > Hey,
>> > 
>> > Ok I've issued a pull request. Please review it 
>> > https://github.com/infinispan/infinispan/pull/418
>> > I am very happy with performance increase observed in new LRU. It 
>> > significantly beats synchronized Map and performance is very close to 
>> > ConcurrentHashMap itself.
>> > I ran tests overnight for 12 hours (each test run is 20 minutes).
>> > 
>> > Dan, please have a look at new MapStressTest and add enhancements you 
>> > discussed with Galder and Sanne. Once you integrate these changes lets 
>> > do another round of testing of both LRU impls and if all goes well we 
>> > can drop old LRU entirely.
>> > 
>> > Cheers,
>> > Vladimir
>> > 
>> > 
>> > 
>> > [ec2-user at ip-10-38-110-25 infinispan]$ ps -e | grep java
>> > [ec2-user at ip-10-38-110-25 infinispan]$ cat perf_new_lru.log
>> > [INFO] Scanning for projects...
>> > [INFO] Reactor build order:
>> > [INFO] Infinispan Common Parent
>> > [INFO] Infinispan Core
>> > [INFO] Infinispan Tools
>> > [INFO] Infinispan Query API
>> > [INFO] Infinispan Tree API
>> > [INFO] Parent pom for cachestore modules
>> > [INFO] Infinispan JDBC CacheStore
>> > [INFO] Infinispan Lucene Directory Implementation
>> > [INFO] Infinispan JDBM CacheStore
>> > [INFO] Infinispan BDBJE CacheStore
>> > [INFO] Infinispan CloudCacheStore
>> > [INFO] Parent pom for server modules
>> > [INFO] Infinispan Server Core Module
>> > [INFO] Infinispan Server Hotrod Module
>> > [INFO] Infinispan Client Hotrod Module
>> > [INFO] Infinispan remote CacheStore
>> > [INFO] Infinispan CassandraCacheStore
>> > [INFO] Infinispan Server Memcached Module
>> > [INFO] Infinispan WebSocket Server
>> > [INFO] Infinispan REST Server
>> > [INFO] Infinispan RHQ Plugin
>> > [INFO] Infinispan Spring Integration
>> > [INFO] Infinispan GUI Demo
>> > [INFO] Infinispan EC2 Demo
>> > [INFO] Infinispan Distributed Executors and Map/Reduce Demo
>> > [INFO] Infinispan EC2 Demo UI
>> > [INFO] Infinispan Directory Demo
>> > [INFO] Infinispan Lucene Directory Demo
>> > [INFO] Infinispan GridFileSystem WebDAV interface
>> > [INFO] Infinispan Distribution
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [INFO] Building Infinispan Common Parent
>> > [INFO] task-segment: [test]
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [INFO] snapshot org.jboss.ws:jbossws-api:1.0.0-SNAPSHOT: checking for 
>> > updates from jboss-public-repository
>> > [INFO] snapshot org.jboss.ws:jbossws-api:1.0.0-SNAPSHOT: checking for 
>> > updates from jboss-public-repository-group
>> > [INFO] snapshot org.jboss.ws:jbossws-parent:1.0.10-SNAPSHOT: checking 
>> > for updates from jboss-public-repository
>> > [INFO] snapshot org.jboss.ws:jbossws-parent:1.0.10-SNAPSHOT: checking 
>> > for updates from jboss-public-repository-group
>> > [INFO] [enforcer:enforce {execution: enforce-java}]
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [INFO] Building Infinispan Core
>> > [INFO] task-segment: [test]
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [INFO] [enforcer:enforce {execution: enforce-java}]
>> > [INFO] [resources:resources {execution: default-resources}]
>> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> > [INFO] Copying 17 resources
>> > [INFO] [compiler:compile {execution: default-compile}]
>> > [INFO] Compiling 52 source files to 
>> > /home/ec2-user/infinispan/core/target/classes
>> > [INFO] Preparing exec:java
>> > [WARNING] Removing: java from forked lifecycle, to prevent recursive 
>> > invocation.
>> > [INFO] [enforcer:enforce {execution: enforce-java}]
>> > [INFO] [exec:java {execution: default}]
>> > Generating schema file in 
>> > /home/ec2-user/infinispan/core/src/main/resources/schema
>> > Using file name infinispan-config-5.0.xsd for schema
>> > Generated schema file successfully
>> > [INFO] [resources:testResources {execution: default-testResources}]
>> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> > [INFO] Copying 24 resources
>> > [INFO] [compiler:testCompile {execution: default-testCompile}]
>> > [INFO] Nothing to compile - all classes are up to date
>> > [INFO] [surefire:test {execution: default-test}]
>> > [INFO] Surefire report directory: 
>> > /home/ec2-user/infinispan/core/target/surefire-reports
>> > 
>> > -------------------------------------------------------
>> > T E S T S
>> > -------------------------------------------------------
>> > Running TestSuite
>> > Performance for container BoundedConcurrentHashMap max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 188
>> > Average put ops/ms 126
>> > Average remove ops/ms 128
>> > Size = 743454
>> > Performance for container BoundedConcurrentHashMap max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 54
>> > Average put ops/ms 33
>> > Average remove ops/ms 41
>> > Size = 762775
>> > Performance for container BoundedConcurrentHashMap max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 26
>> > Average put ops/ms 15
>> > Average remove ops/ms 19
>> > Size = 758876
>> > [testng-MapStressTest] Test 
>> > testBufferedConcurrentHashMapLRU(org.infinispan.stress.MapStressTest) 
>> > succeeded.
>> > Test suite progress: tests succeeded: 1, failed: 0, skipped: 0.
>> > Performance for container BoundedConcurrentHashMap max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 305
>> > Average put ops/ms 99
>> > Average remove ops/ms 213
>> > Size = 98200
>> > Performance for container BoundedConcurrentHashMap max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 89
>> > Average put ops/ms 30
>> > Average remove ops/ms 55
>> > Size = 96159
>> > Performance for container BoundedConcurrentHashMap max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 47
>> > Average put ops/ms 12
>> > Average remove ops/ms 19
>> > Size = 96977
>> > [testng-MapStressTest] Test 
>> > testBufferedConcurrentHashMapLRU(org.infinispan.stress.MapStressTest) 
>> > succeeded.
>> > Test suite progress: tests succeeded: 2, failed: 0, skipped: 0.
>> > Performance for container BoundedConcurrentHashMap max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 234
>> > Average put ops/ms 112
>> > Average remove ops/ms 158
>> > Size = 393215
>> > Performance for container BoundedConcurrentHashMap max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 68
>> > Average put ops/ms 32
>> > Average remove ops/ms 47
>> > Size = 391512
>> > Performance for container BoundedConcurrentHashMap max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 35
>> > Average put ops/ms 15
>> > Average remove ops/ms 21
>> > Size = 390856
>> > [testng-MapStressTest] Test 
>> > testBufferedConcurrentHashMapLRU(org.infinispan.stress.MapStressTest) 
>> > succeeded.
>> > Test suite progress: tests succeeded: 3, failed: 0, skipped: 0.
>> > Performance for container CacheImpl max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 104
>> > Average put ops/ms 48
>> > Average remove ops/ms 58
>> > Size = 738030
>> > Performance for container CacheImpl max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 30
>> > Average put ops/ms 13
>> > Average remove ops/ms 22
>> > Size = 774764
>> > Performance for container CacheImpl max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 16
>> > Average put ops/ms 6
>> > Average remove ops/ms 13
>> > Size = 776668
>> > [testng-MapStressTest] Test 
>> > testCache(org.infinispan.stress.MapStressTest) succeeded.
>> > Test suite progress: tests succeeded: 4, failed: 0, skipped: 0.
>> > Performance for container CacheImpl max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 220
>> > Average put ops/ms 44
>> > Average remove ops/ms 123
>> > Size = 98304
>> > Performance for container CacheImpl max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 66
>> > Average put ops/ms 12
>> > Average remove ops/ms 39
>> > Size = 97832
>> > Performance for container CacheImpl max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 34
>> > Average put ops/ms 5
>> > Average remove ops/ms 21
>> > Size = 97802
>> > [testng-MapStressTest] Test 
>> > testCache(org.infinispan.stress.MapStressTest) succeeded.
>> > Test suite progress: tests succeeded: 5, failed: 0, skipped: 0.
>> > Performance for container CacheImpl max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 144
>> > Average put ops/ms 44
>> > Average remove ops/ms 77
>> > Size = 393216
>> > Performance for container CacheImpl max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 42
>> > Average put ops/ms 12
>> > Average remove ops/ms 28
>> > Size = 392794
>> > Performance for container CacheImpl max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 22
>> > Average put ops/ms 6
>> > Average remove ops/ms 16
>> > Size = 387007
>> > [testng-MapStressTest] Test 
>> > testCache(org.infinispan.stress.MapStressTest) succeeded.
>> > Test suite progress: tests succeeded: 6, failed: 0, skipped: 0.
>> > Performance for container ConcurrentHashMap max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 303
>> > Average put ops/ms 235
>> > Average remove ops/ms 240
>> > Size = 711798
>> > Performance for container ConcurrentHashMap max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 88
>> > Average put ops/ms 68
>> > Average remove ops/ms 70
>> > Size = 669381
>> > Performance for container ConcurrentHashMap max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 45
>> > Average put ops/ms 29
>> > Average remove ops/ms 31
>> > Size = 758993
>> > [testng-MapStressTest] Test 
>> > testConcurrentHashMap(org.infinispan.stress.MapStressTest) succeeded.
>> > Test suite progress: tests succeeded: 7, failed: 0, skipped: 0.
>> > Performance for container ConcurrentHashMap max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 261
>> > Average put ops/ms 198
>> > Average remove ops/ms 197
>> > Size = 667905
>> > Performance for container ConcurrentHashMap max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 75
>> > Average put ops/ms 55
>> > Average remove ops/ms 56
>> > Size = 614548
>> > Performance for container ConcurrentHashMap max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 37
>> > Average put ops/ms 24
>> > Average remove ops/ms 25
>> > Size = 789004
>> > [testng-MapStressTest] Test 
>> > testConcurrentHashMap(org.infinispan.stress.MapStressTest) succeeded.
>> > Test suite progress: tests succeeded: 8, failed: 0, skipped: 0.
>> > Performance for container ConcurrentHashMap max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 305
>> > Average put ops/ms 231
>> > Average remove ops/ms 236
>> > Size = 656558
>> > Performance for container ConcurrentHashMap max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 88
>> > Average put ops/ms 66
>> > Average remove ops/ms 68
>> > Size = 716811
>> > Performance for container ConcurrentHashMap max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 44
>> > Average put ops/ms 30
>> > Average remove ops/ms 31
>> > Size = 757864
>> > [testng-MapStressTest] Test 
>> > testConcurrentHashMap(org.infinispan.stress.MapStressTest) succeeded.
>> > Test suite progress: tests succeeded: 9, failed: 0, skipped: 0.
>> > Performance for container SynchronizedMap max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 171
>> > Average put ops/ms 171
>> > Average remove ops/ms 142
>> > Size = 679283
>> > Performance for container SynchronizedMap max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 52
>> > Average put ops/ms 52
>> > Average remove ops/ms 38
>> > Size = 806142
>> > Performance for container SynchronizedMap max capacity is 
>> > 1048576[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 26
>> > Average put ops/ms 27
>> > Average remove ops/ms 19
>> > Size = 860559
>> > [testng-MapStressTest] Test 
>> > testHashMap(org.infinispan.stress.MapStressTest) succeeded.
>> > Test suite progress: tests succeeded: 10, failed: 0, skipped: 0.
>> > Performance for container SynchronizedMap max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 171
>> > Average put ops/ms 172
>> > Average remove ops/ms 151
>> > Size = 722315
>> > Performance for container SynchronizedMap max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 51
>> > Average put ops/ms 54
>> > Average remove ops/ms 47
>> > Size = 810163
>> > Performance for container SynchronizedMap max capacity is 
>> > 131072[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 25
>> > Average put ops/ms 26
>> > Average remove ops/ms 22
>> > Size = 848856
>> > [testng-MapStressTest] Test 
>> > testHashMap(org.infinispan.stress.MapStressTest) succeeded.
>> > Test suite progress: tests succeeded: 11, failed: 0, skipped: 0.
>> > Performance for container SynchronizedMap max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[8,2,1]
>> > Average get ops/ms 169
>> > Average put ops/ms 173
>> > Average remove ops/ms 151
>> > Size = 802294
>> > Performance for container SynchronizedMap max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[32,4,2]
>> > Average get ops/ms 51
>> > Average put ops/ms 53
>> > Average remove ops/ms 47
>> > Size = 666371
>> > Performance for container SynchronizedMap max capacity is 
>> > 524288[numReaders,numWriters,numRemovers]=[64,8,3]
>> > Average get ops/ms 26
>> > Average put ops/ms 26
>> > Average remove ops/ms 21
>> > Size = 829093
>> > [testng-MapStressTest] Test 
>> > testHashMap(org.infinispan.stress.MapStressTest) succeeded.
>> > Test suite progress: tests succeeded: 12, failed: 0, skipped: 0.
>> > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
>> > 43,209.511 sec
>> > 
>> > Results :
>> > 
>> > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
>> > 
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [INFO] Building Infinispan Tools
>> > [INFO] task-segment: [test]
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [INFO] snapshot org.infinispan:infinispan-core:5.0.0-SNAPSHOT: checking 
>> > for updates from jboss-public-repository
>> > [INFO] [enforcer:enforce {execution: enforce-java}]
>> > [INFO] [resources:resources {execution: default-resources}]
>> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> > [INFO] Copying 2 resources
>> > [INFO] [compiler:compile {execution: default-compile}]
>> > [INFO] Nothing to compile - all classes are up to date
>> > [INFO] [resources:testResources {execution: default-testResources}]
>> > [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> > [INFO] skip non existing resourceDirectory 
>> > /home/ec2-user/infinispan/tools/src/test/resources
>> > [INFO] [compiler:testCompile {execution: default-testCompile}]
>> > [INFO] Nothing to compile - all classes are up to date
>> > [INFO] [surefire:test {execution: default-test}]
>> > [INFO] Surefire report directory: 
>> > /home/ec2-user/infinispan/tools/target/surefire-reports
>> > 
>> > -------------------------------------------------------
>> > T E S T S
>> > -------------------------------------------------------
>> > There are no tests to run.
>> > 
>> > Results :
>> > 
>> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
>> > 
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [ERROR] BUILD FAILURE
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [INFO] No tests were executed! (Set -DfailIfNoTests=false to ignore 
>> > this error.)
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [INFO] For more information, run Maven with the -e switch
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [INFO] Total time: 720 minutes 23 seconds
>> > [INFO] Finished at: Wed Jul 06 17:13:51 UTC 2011
>> > [INFO] Final Memory: 206M/2001M
>> > [INFO] 
>> > ------------------------------------------------------------------------
>> > [ec2-user at ip-10-38-110-25 infinispan]$
>> > _______________________________________________
>> > infinispan-dev mailing list
>> > infinispan-dev at lists.jboss.org
>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> 
>> 
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

--
Manik Surtani
manik at jboss.org
twitter.com/maniksurtani

Lead, Infinispan
http://www.infinispan.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20110707/70566b8f/attachment-0001.html 