[infinispan-issues] [JBoss JIRA] (ISPN-6706) Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled

[Krzysztof Sobolewski (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242530#comment-13242530 ] 

Krzysztof Sobolewski commented on ISPN-6706:
--------------------------------------------

[~wburns] says:

> Unfortunately CacheWriter purge isn't related to eviction.  This is to remove entries that have expired. Expiration can be configured both via xml and inserted programatically.  Due to the latter we have to always call the purge method.  It is feasible to keep state on the writer such as a boolean to trigger if any entry has ever been able to expire and it not then skip purge.

I know that expiration is not the same as eviction, I'm not cofusing these two :) This simple change works for my use case and I know it's not general solution. But I think the optimization is important - the expiration purge really puts a heavy load on the database!

> Purging cache writers is [mostly] redundant when eviction is disabled and preload is enabled
> --------------------------------------------------------------------------------------------
>
>                 Key: ISPN-6706
>                 URL: https://issues.jboss.org/browse/ISPN-6706
>             Project: Infinispan
>          Issue Type: Enhancement
>          Components: Loaders and Stores
>    Affects Versions: 8.2.2.Final
>            Reporter: Krzysztof Sobolewski
>
> This issue arised when I was testing a cluster with about 16 million entries. Our configuration is that all the data is also kept in memory, so eviction is disabled in this cache. But expiration is enabled. During the test I noticed pauses that started small but increased while the test was progressing, reaching more than 20 seconds at one point. After ruling out maintenance tasks in MySQL that could interfere, I discovered that the pause is caused by the expiration thread purging the database for expired entries. This was a huge and unnecessary drag so I hacked Infinispan to skip the purge of persistent state in cases when it's likely to be redundant with purging the transient state. I say "likely" because entries evicted maually via the evict() call poke a huge hole in the underlying assumptions :) Anyway, our cluster no longer regularly pauses for half a minute, so here's something for your consideration.



--
This message was sent by Atlassian JIRA
(v6.4.11#64026)