[
https://issues.jboss.org/browse/ISPN-6706?page=com.atlassian.jira.plugin....
]
William Burns edited comment on ISPN-6706 at 5/25/16 9:42 AM:
--------------------------------------------------------------
If you are running a cluster I just noticed we don't have a very simple optimization
of only running purge on the coordinator node when a shared store is used. This means
that without this all nodes may be hitting the DB server at approximately the same time,
which is probably why it is killing your DB response times. Another thing I would suggest
is to limit how often the cache writer purge is performed in comparison to in memory (it
could be ran every 5th memory purge for example).
And looking closer at the JPAStore and Jdbc*Store implementations they should be using a
simple select that only returns expired entries. I am not sure if we have indexes on
these fields or not though, I haven't looked too closely at these classes (which may
be causing a full table scan).
Also the optimization you have right now could still be used but we would have to limit it
as the following:
# Only supported with 1 cache loader/writer (note we can have multiple configured)
# Store state in the persistence manager to guarantee evict was never called
was (Author: william.burns):
If you are running a cluster I just noticed we don't have a very simple optimization
of only running purge on the coordinator node when a shared store is used. This means
that without this all nodes may be hitting the DB server at approximately the same time,
which is probably why it is killing your DB response times. Another thing I would suggest
is to limit how often the cache writer purge is performed in comparison to in memory (it
could be ran every 5th memory purge for example).
And looking closer at the JPAStore and Jdbc*Store implementations they should be using a
simple select that only returns expired entries. I am not sure if we have indexes on
these fields or not though, I haven't looked too closely at these classes (which may
be causing your full table scan).
Also the optimization you have right now could still be used but we would have to limit it
as the following:
# Only supported with 1 cache loader/writer (note we can have multiple configured)
# Store state in the persistence manager to guarantee evict was never called
Purging cache writers is [mostly] redundant when eviction is disabled
and preload is enabled
--------------------------------------------------------------------------------------------
Key: ISPN-6706
URL:
https://issues.jboss.org/browse/ISPN-6706
Project: Infinispan
Issue Type: Enhancement
Components: Loaders and Stores
Affects Versions: 8.2.2.Final
Reporter: Krzysztof Sobolewski
This issue arised when I was testing a cluster with about 16 million entries. Our
configuration is that all the data is also kept in memory, so eviction is disabled in this
cache. But expiration is enabled. During the test I noticed pauses that started small but
increased while the test was progressing, reaching more than 20 seconds at one point.
After ruling out maintenance tasks in MySQL that could interfere, I discovered that the
pause is caused by the expiration thread purging the database for expired entries. This
was a huge and unnecessary drag so I hacked Infinispan to skip the purge of persistent
state in cases when it's likely to be redundant with purging the transient state. I
say "likely" because entries evicted maually via the evict() call poke a huge
hole in the underlying assumptions :) Anyway, our cluster no longer regularly pauses for
half a minute, so here's something for your consideration.
--
This message was sent by Atlassian JIRA
(v6.4.11#64026)