<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On 8 Jan 2010, at 12:35, Bryan Thompson wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<div style="WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space">
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">Vladimir,</font></span></div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">Reading the paper, it appears that my suggestion of
segments is called "distributed locks" in this paper and the authors
suggests:</font></span></div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font size="2"></font></span> </div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font size="2">> For
example, the algorithms<span class="661490010-08012010"> </span>that need to
detect sequence of accesses cannot retain their<span class="661490010-08012010">
</span>performance advantages when pages in the same sequence<span class="661490010-08012010"> </span>have been distributed into multiple partitions
and cannot be<span class="661490010-08012010"> </span>identified as a
sequence.</font></span></div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">Overall, I like the approach outlined in the paper.
It takes the concept of batching further than what we did, but in much the same
spirit. In our implementation, we only batched evictions. They are
also batching the updates to the replacement policy ordering which is still a
bottleneck for us.</font></span></div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">Concerning the FIFO queue, t</font></span><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">he paper
(page 372) considers both per-thread queues or global queues. They dismiss
global queues for batching updates primarily because certain cache replacement
policies need to detect sequences of operations. Can such sequences of
operations be meaningfully detected within infinispan? Is there any
thread-affinity for a transaction?</font></span></div></div></blockquote><div><br></div><div>No there isn't - although there could be, as a side effect of certain use cases and access patterns, but this should not be expected.</div><br><blockquote type="cite"><div style="WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space"><div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4"> If not, then segmented queues might be
an alternative for infinispan.</font></span></div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">Also, for java, i</font></span><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">f the
hash map uses weak values then cache entries then any entries on the FIFO queues
will remain pinned in the hash map even if they would have otherwise been
evicted by the cache policy which is a nice touch. That way you can
continue to access an object which remains accessible to the JVM held even if
the cache replacement policy would have discarded the cache
entry.</font></span></div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">The experiment in the paper compared several cache
replacement policies, but the data were all hot in cache and the cache was large
enough to avoid all cache misses, so I am not clear why they bothered to compare
different cache replacement policies -- none of them did any evictions for their
tests. </font></span><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">While "prefetching" did reduce the lock
contention slightly, it had no meaningful effect on either throughput or
response time. The authors posit that it might have more effect on
machines with many more cores, depending on the CPU architecture. It seems
easily enough to ignore the prefetch dimension for now and just concentrate on
batching.</font></span></div></div></blockquote><div><br></div><div>+1</div><br><blockquote type="cite"><div style="WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space">
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">So, I think that the batching approach per this paper makes
sense and could be combined with LRU or LIRS easily since the cache replacement
policy is encapsulated (which might not be true with prefetch) and seems overall
like a good idea and an improvement on what we were already doing. For our
application, I think that we would use a hash map with weak values (per above)
to gain the benefit of the actual retention time of the object by the JVM.
I would also like to explore the use of segmented queues as well as per-thread
queues to control the #of queues in the system separately from the #of
threads. By batching updates to the access order, you are also effectively
batching evictions since they are driven by updates. When combined with a
weak value caches, the thread running the updates should also clear entries from
the cache on the weak reference queue, thereby batching the clearing of stale
cache entries (those with cleared references) at the same time it is batching
updates to the access order and inserts to the cache.</font></span></div>
<div dir="ltr" align="left"><span class="661490010-08012010"></span><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="661490010-08012010"><font face="Courier New" color="#0000ff" size="4">Bryan</font></span></div><br>
<blockquote dir="ltr" style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<div class="OutlookMessageHeader" lang="en-us" dir="ltr" align="left">
<hr tabindex="-1">
<font face="Tahoma" size="2"><b>From:</b> <a href="mailto:infinispan-dev-bounces@lists.jboss.org">infinispan-dev-bounces@lists.jboss.org</a>
[mailto:infinispan-dev-bounces@lists.jboss.org] <b>On Behalf Of </b>Vladimir
Blagojevic<br><b>Sent:</b> Friday, January 08, 2010 4:42 AM<br><b>To:</b>
infinispan -Dev List<br><b>Subject:</b> Re: [infinispan-dev] Implementing LIRS
eviction policy [ISPN-299]<br></font><br></div>
<div></div>Hey Bryan,
<div><br>
<div>
<div>On 2010-01-08, at 2:09 AM, Bryan Thompson wrote:</div><br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space">
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4">Valdimir,</font></span></div>
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4">I have used a similar approach to
"batching" evictions with good success. It reduced contention for the
lock to handle eviction by about twenty
percent.</font></span></div></div></blockquote>
<div><br></div>
<div>How different, if at all, was your batching implementation from the one I
referred to in my previous mail? </div><br>
<blockquote type="cite">
<div style="WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space">
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4">For this cache, which was not
realized using infinispan, I did NO eviction until the cache reached a
target percentage of the JVM heap (this kept the cache hot). For our
case, this was easy as the objects entered in the cache were database
records coded as byte[]s so have a good measure for their memory
footprint. Once adding another record to the cache would push the
cache size over the target memory limit, that thread would gain a lock and
then evict a batch of records (in an LRU fashion) until the cache size had
been driven down by a threashold (e.g., 5% of the target memory was
recovered).</font></span></div></div></blockquote>
<blockquote type="cite">
<div style="WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space">
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4">Let me suggest an alternative
approach. Neither LIRS nor LRU need to be strong orderings. It
is enough that a cache eviction policy be "approximately" LRU (or
LIRS). As long as the least desirable entries wind up getting evicted,
it does not matter what the precise eviction order
is.</font></span></div></div></blockquote>
<div><br></div>
<div>Can you clarify "need to be strong orderings"? Aha, you are referring to
eviction order? </div><br>
<blockquote type="cite">
<div style="WORD-WRAP: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space">
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4">If you accept "approximate" eviction
orders, then striped locks could be used to reduce thread contention.
Evictions could be driven on each segment of the cache independently.
If we hash the cache entry key to choose the segment, then each cache entry
will only appear in one segment and all segments should be driven more or
less equally by the application workload (assuming a decent hash
function). So using 16 segments / locks, you have 1/16th the
contention.</font></span></div>
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4"></font></span> </div>
<div dir="ltr" align="left"><span class="427114300-08012010"><font face="Courier New" color="#0000ff" size="4">Thoughts?</font></span></div></div></blockquote></div><br>
<div>So what you are basically saying is that we implement LIRS and LRU per
Segment of ConcurrentHashMap, is that right? Interesting
idea! </div>
<div><br></div>
<div>Does your proposal still contain batching? If so, how would you fit it in
this new design? You would have to keep batched record of cache entry accesses
in a queue per segment? </div>
<div><br></div>
<div>Regards,</div>
<div>Vladimir</div>
<div><br></div>
<div><br></div></div></blockquote></div>
_______________________________________________<br>infinispan-dev mailing list<br><a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>https://lists.jboss.org/mailman/listinfo/infinispan-dev</blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>--</div><div>Manik Surtani</div><div><a href="mailto:manik@jboss.org">manik@jboss.org</a></div><div>Lead, Infinispan</div><div>Lead, JBoss Cache</div><div><a href="http://www.infinispan.org">http://www.infinispan.org</a></div><div><a href="http://www.jbosscache.org">http://www.jbosscache.org</a></div><div><br></div></div></span><br class="Apple-interchange-newline"></span><br class="Apple-interchange-newline">
</div>
<br></body></html>