[infinispan-issues] [JBoss JIRA] Commented: (ISPN-663) Eviction with passivation using JdbmCacheStore is 100 times slower in 4.1 vs 4.0

[Trustin Lee (JIRA)]

    [ https://jira.jboss.org/browse/ISPN-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556515#action_12556515 ] 

Trustin Lee commented on ISPN-663:
----------------------------------

Re 1) Great.
Re 2) There's no alternative with such a nice license unfortunately.  It might be a good idea to borrow some code from other higher level libraries like Derby and Lucene, because they should be implementing HTree or BTree somehow.  JDBM is way too old and we should have a decent disk-based CacheStore implementation optimized for Infinispan to compete with others in terms of performance.  We will be able to beat them much easily in a single node scenario with it - others usually have their own optimized storage implementation.

Loading happens both in 4.0 and 4.2.  The difference is that passivation happens at much later point in 4.0.  In 4.0:

1) (Main thread) Measurement begins
2) (Main thread) Lots of load operations as a side effect of put()s.
3) (Main thread) Measurement ends
4) (Passivation thread) Lots of delayed store operations

in 4.2 w/o AsyncStore:

1) (Main thread) Measurement begins
2) (Main thread) Lots of load() and store() operations as a side effect of put()s.
3) (Main thread) Measurement ends

in 4.2 w/ AsyncStore:

1) (Main thread) Measurement begins
2-a) (Main thread) Lots of load() operations as a side effect of put()s.
2-b) (Async thread) Lots of put() operations (not delayed, runs simultaneously with 2-a)
3) (Main thread) Measurement ends
4) (Async thread) Small number of unfinished put() operations

So, the test itself is somewhat flawed, but it shows us a couple points of optimization.

> Eviction with passivation using JdbmCacheStore is 100 times slower in 4.1 vs 4.0
> --------------------------------------------------------------------------------
>
>                 Key: ISPN-663
>                 URL: https://jira.jboss.org/browse/ISPN-663
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Eviction, Loaders and Stores
>    Affects Versions: 4.1.0.Final
>         Environment: Win32 JRE 1.6.0_21
>            Reporter: Paul Nardone
>            Assignee: Trustin Lee
>             Fix For: 4.2.0.BETA1, 4.2.0.Final
>
>         Attachments: InfinispanPassivationTest.java
>
>
> Eviction with passivation enabled using the JdbmCacheStore appears to be significantly slower in 4.1.0.FINAL vs 4.0.0.FINAL.
> The degredation in performance is so signficant to make it impossible  to use
> The  performance issue seems to due as the JdbmCacheStore synching the filesystem via FileDescriptor.sync() or similar which occurs during every object passivation and each passivation occurs as a new object is added beyond the EvictionMaxEntries capacity.
> The attached test inserts 1000 values into two caches
> Both caches use a JdbmCacheStore and  LRU
> PASSIVATIONLRU10 runs with cache with EvictionMaxEntries  10
> PASSIVATIONLRU1000 runs with cache with EvictionMaxEntries  1000
> 4.1.0.FINAL
> PASSIVATIONLRU10 Time Taken : 51704
> PASSIVATIONLRU1000 Time Taken : 4484
> 4.0.0.FINAL
> PASSIVATIONLRU10 Time Taken : 281
> PASSIVATIONLRU1000 Time Taken : 141
> 4.2.0.ALPHA2
> PASSIVATIONLRU10 Time Taken : 51047
> PASSIVATIONLRU1000 Time Taken : 5156

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira