[infinispan-issues] [JBoss JIRA] Commented: (ISPN-663) Eviction with passivation using JdbmCacheStore is 100 times slower in 4.1 vs 4.0

[Trustin Lee (JIRA)]

    [ https://jira.jboss.org/browse/ISPN-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12555208#action_12555208 ] 

Trustin Lee commented on ISPN-663:
----------------------------------

1)

The current DefaultDataContainer implementation seems to implement PIGGYBACK thread policy only, and that's probably why the DEFAULT thread policy doesn't trigger the eviction to be executed in a separate thread, hence passivation always piggybacked.  Also, Manik said that eviction since 4.1 doesn't use a background thread.  Then is DEFAULT thread policy obsolete now? (i.e. always PIGGYBACK)  I guess so.

The resolution I can think of is to improve DefaultDataContainer and BoundedConcurrentHashMap so that we can resurrect the old behavior (DEFAULT), but in a safe way (i.e. block put() operation if passivation backlog is too large).  Maybe we could make PIGGYBACK default and introduce a new thread policy called BACKGROUND.  DEFAULT could refer to PIGGYBACK or BACKGROUND instead.

2)

On the other side, a unfinished JDBM write can be lost if commit() was not called yet, just like JDBC.  Therefore, commit() must be called for any single store() operation.  commit() can be delayed only when we are sure the entries associated with the previous uncommitted store() operations were not evicted from the memory yet - the current CacheStore interface is not given with such hint.

Adding CacheStore.store(InternalCacheEntry[]) will give the CacheStore enough hint, but it will result in an additional API change - EvictionListener.postEvict() should be changed to accept multiple entries. (e.g. postEvict(Object[] keys, InternalCacheEntry[] values)), which means broken backward compatibility.  I think we should go with this option in 5.0 because there is obvious advantage in performance with most CacheStores.

WDYT about (1) and (2)?

> Eviction with passivation using JdbmCacheStore is 100 times slower in 4.1 vs 4.0
> --------------------------------------------------------------------------------
>
>                 Key: ISPN-663
>                 URL: https://jira.jboss.org/browse/ISPN-663
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Eviction, Loaders and Stores
>    Affects Versions: 4.1.0.Final
>         Environment: Win32 JRE 1.6.0_21
>            Reporter: Paul Nardone
>            Assignee: Trustin Lee
>             Fix For: 4.2.0.BETA1, 4.2.0.Final
>
>         Attachments: InfinispanPassivationTest.java
>
>
> Eviction with passivation enabled using the JdbmCacheStore appears to be significantly slower in 4.1.0.FINAL vs 4.0.0.FINAL.
> The degredation in performance is so signficant to make it impossible  to use
> The  performance issue seems to due as the JdbmCacheStore synching the filesystem via FileDescriptor.sync() or similar which occurs during every object passivation and each passivation occurs as a new object is added beyond the EvictionMaxEntries capacity.
> The attached test inserts 1000 values into two caches
> Both caches use a JdbmCacheStore and  LRU
> PASSIVATIONLRU10 runs with cache with EvictionMaxEntries  10
> PASSIVATIONLRU1000 runs with cache with EvictionMaxEntries  1000
> 4.1.0.FINAL
> PASSIVATIONLRU10 Time Taken : 51704
> PASSIVATIONLRU1000 Time Taken : 4484
> 4.0.0.FINAL
> PASSIVATIONLRU10 Time Taken : 281
> PASSIVATIONLRU1000 Time Taken : 141
> 4.2.0.ALPHA2
> PASSIVATIONLRU10 Time Taken : 51047
> PASSIVATIONLRU1000 Time Taken : 5156

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira