[infinispan-issues] [JBoss JIRA] Commented: (ISPN-663) Eviction with passivation using JdbmCacheStore is 100 times slower in 4.1 vs 4.0

[Trustin Lee (JIRA)]

    [ https://jira.jboss.org/browse/ISPN-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556307#action_12556307 ] 

Trustin Lee commented on ISPN-663:
----------------------------------

That's a very good question.  I was actually wondering why, too. :)

When Cache.put() is called in Paul's test, the stack trace always looks like this:

	at org.infinispan.loaders.jdbm.JdbmCacheStore.load(JdbmCacheStore.java:135)
	at org.infinispan.loaders.decorators.AbstractDelegatingStore.load(AbstractDelegatingStore.java:86)
	at org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:138)
	at org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(CacheLoaderInterceptor.java:78)
	at org.infinispan.interceptors.ActivationInterceptor.visitPutKeyValueCommand(ActivationInterceptor.java:37)
	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
	at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:57)
	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:172)
	at org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:120)
	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
	at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:87)
	at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:58)
	at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:57)
	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:273)
	at org.infinispan.CacheDelegate.put(CacheDelegate.java:427)
	at org.infinispan.CacheSupport.put(CacheSupport.java:28)
	at test.InfinispanPassivationTest.runTest(InfinispanPassivationTest.java:108)

When passivation triggers CacheStore.store(), with AsyncStore enabled, the stack trace looks like the following:

	at org.infinispan.loaders.jdbm.JdbmCacheStore.store0(JdbmCacheStore.java:300)
	at org.infinispan.loaders.jdbm.JdbmCacheStore.store(JdbmCacheStore.java:278)
	at org.infinispan.loaders.decorators.AbstractDelegatingStore.store(AbstractDelegatingStore.java:46)
	at org.infinispan.loaders.decorators.AsyncStore.applyModificationsSync(AsyncStore.java:205)
	at org.infinispan.loaders.decorators.AsyncStore$AsyncProcessor.put(AsyncStore.java:372)
	at org.infinispan.loaders.decorators.AsyncStore$AsyncProcessor.innerRun(AsyncStore.java:355)
	at org.infinispan.loaders.decorators.AsyncStore$AsyncProcessor.run(AsyncStore.java:273)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:619)

There are two issues:

1) A put() operation results in one load() and one store() operation.  Could it be optimized?  Instead of loading from CacheStore first, the entry could be stored to the CacheStore first and then loaded back? (e.g. by introducing CacheStore.storeAndLoad())

2) Since JDBM is not designed to be highly concurrent, HTree.get() and HTree.put() are being guarded by the same lock.  That is why using AsyncStore doesn't simply improve the cache operation time.  The viable solution would probably be writing a better HTree implementation and ditch JDBM?  It would be nice if BDBJE has less restrictive license, but that's not going to happen soon.

> Eviction with passivation using JdbmCacheStore is 100 times slower in 4.1 vs 4.0
> --------------------------------------------------------------------------------
>
>                 Key: ISPN-663
>                 URL: https://jira.jboss.org/browse/ISPN-663
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Eviction, Loaders and Stores
>    Affects Versions: 4.1.0.Final
>         Environment: Win32 JRE 1.6.0_21
>            Reporter: Paul Nardone
>            Assignee: Trustin Lee
>             Fix For: 4.2.0.BETA1, 4.2.0.Final
>
>         Attachments: InfinispanPassivationTest.java
>
>
> Eviction with passivation enabled using the JdbmCacheStore appears to be significantly slower in 4.1.0.FINAL vs 4.0.0.FINAL.
> The degredation in performance is so signficant to make it impossible  to use
> The  performance issue seems to due as the JdbmCacheStore synching the filesystem via FileDescriptor.sync() or similar which occurs during every object passivation and each passivation occurs as a new object is added beyond the EvictionMaxEntries capacity.
> The attached test inserts 1000 values into two caches
> Both caches use a JdbmCacheStore and  LRU
> PASSIVATIONLRU10 runs with cache with EvictionMaxEntries  10
> PASSIVATIONLRU1000 runs with cache with EvictionMaxEntries  1000
> 4.1.0.FINAL
> PASSIVATIONLRU10 Time Taken : 51704
> PASSIVATIONLRU1000 Time Taken : 4484
> 4.0.0.FINAL
> PASSIVATIONLRU10 Time Taken : 281
> PASSIVATIONLRU1000 Time Taken : 141
> 4.2.0.ALPHA2
> PASSIVATIONLRU10 Time Taken : 51047
> PASSIVATIONLRU1000 Time Taken : 5156

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira