<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Am 04.10.11 20:15, schrieb Mircea Markus:
<blockquote
cite="mid:853438B0-56A1-4211-BF31-403DAA9C876F@jboss.com"
type="cite">
<div>
<div>On 15 Sep 2011, at 17:50, Olaf Bergner wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
<div bgcolor="#FFFFFF" text="#000000"> Hello,<br>
<br>
it's been a while ... I'm the guy who a few months ago
implemented Infinispan's Spring support and started to work
on ISPN-78 - Large Object Support. Due to very personal
reasons and a job change that required my undivided
attention that latter endeavour went into a hiatus. Sorry
for that.<br>
<br>
Provided there's still interest in the community I'd like to
continue working on ISPN-78. But first let's see where we
stand today. What follows is in large part a reiteration of
what was already said a few months ago, but not all of us
are blessed with an infallible memory. <br>
<br>
At <a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://github.com/obergner/infinispan/tree/t_ispn78">https://github.com/obergner/infinispan/tree/t_ispn78</a>
you will find a very simple solution based on the concept of
a StreamingHandler, to be obtained via
Cache.getStreamingHandler():<br>
<br>
<meta charset="utf-8">
<span class="Apple-style-span" style="font-family:
helvetica, arial, freesans, clean, sans-serif; font-size:
11px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height:
14px; orphans: 2; text-align: -webkit-auto; text-indent:
0px; text-transform: none; white-space: normal; widows: 2;
word-spacing: 0px; -webkit-text-decorations-in-effect:
none; -webkit-text-size-adjust: auto;
-webkit-text-stroke-width: 0px; background-color: rgb(255,
255, 255); ">
<pre style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font: normal normal normal 12px/normal 'Bitstream Vera Sans Mono', Courier, monospace; font-family: 'Bitstream Vera Sans Mono', Courier, monospace; font-size: 12px; line-height: 1.4; "><div class="line" id="LC42" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; "><span class="kd" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">public</span> <span class="kd" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">interface</span> <span class="
nc" styl="" e="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; color: rgb(68, 85, 136); font-weight: bold; ">StreamingHandler</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; "><</span><span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">K</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">></span> <span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px
; padding-le
ft: 0px; font-weight: bold; ">{</span></div><div class="line" id="LC43" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; ">
</div><div class="line" id="LC59" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; "> <span class="kt" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; color: rgb(68, 85, 136); font-weight: bold; ">void</span> <span class="nf" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; color: rgb(153, 0, 0); font-weight: bold; ">writeToKey</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">(</span><span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-to
p: 0px; pa
dding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">K</span> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">key</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">,</span> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">InputStream</span> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">largeObject</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; pad
ding-bottom: 0px; padding-left: 0px; font-weight: bold; ">);</span></div><div class="line" id="LC60" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; ">
</div><div class="line" id="LC76" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; "> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">OutputStream</span> <span class="nf" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; color: rgb(153, 0, 0); font-weight: bold; ">writeToKey</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">(</span><span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-b
ottom: 0px
; padding-left: 0px; ">K</span> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">key</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">);</span></div><div class="line" id="LC77" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; ">
</div><div class="line" id="LC91" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; "> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">InputStream</span> <span class="nf" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; color: rgb(153, 0, 0); font-weight: bold; ">readFromKey</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">(</span><span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-b
ottom: 0px
; padding-left: 0px; ">K</span> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">key</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">);</span></div><div class="line" id="LC92" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; ">
</div><div class="line" id="LC103" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; "> <span class="kt" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; color: rgb(68, 85, 136); font-weight: bold; ">boolean</span> <span class="nf" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; color: rgb(153, 0, 0); font-weight: bold; ">removeKey</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">(</span><span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding
-top: 0px;
padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">K</span> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">key</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">);</span></div><div class="line" id="LC104" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; ">
</div><div class="line" id="LC119" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; "> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">StreamingHandler</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; "><</span><span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">K</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: b
old; ">>
;</span> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">withFlags</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">(</span><span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">Flag</span><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">...</span> <span class="n" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; ">flags</span>
<span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">);</span></div><div class="line" id="LC120" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 1em; "><span class="o" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; font-weight: bold; ">}</span></div></pre>
</span><br>
This solution basically works yet needs some love before it
could be left out into the wild. I think it's not too bad
but in the interim I have come to prefer a different
solution I would like to start working on, namely a
dedicated StreamingCache that's completely separate from the
regular Cache.<br>
</div>
</blockquote>
I guess you'd obtain a StreamingCache from the CacheManager
rather than obtaining it from a Cache? <br>
</div>
</blockquote>
Yes, that's my current line of thinking, at least.<br>
<blockquote
cite="mid:853438B0-56A1-4211-BF31-403DAA9C876F@jboss.com"
type="cite">
<div>
<blockquote type="cite">
<div bgcolor="#FFFFFF" text="#000000"> <br>
Pros:<br>
<br>
1. I think that it's highly unlikely that users would want
store large objects as well as "regular" objects within the
same cache instance. In my experience these two classes of
objects are treated entirely differently on the application
level. <br>
<br>
2. Moreover I *suspect* - though I know next to nothing
about these matters - that a user would have a hard time
finding a set of tuning parameters to satisfy the needs of
both regular as well as large objects.<br>
</div>
</blockquote>
<blockquote type="cite">
<div bgcolor="#FFFFFF" text="#000000"><font
class="Apple-style-span" color="#000000"><br>
</font> 3. Keeping large object support as part of the
regular cache *might* entail that the same code paths would
have to be optimized for both regular as well as large
objects. This *could* prove difficult in the long term.<br>
<br>
4. A separate StreamingCache would open up the possibility
of defining default settings tailored to the needs of large
objects. Furthermore I *suspect* that many cache settings
would be irrelevant for large objects.<br>
<br>
5. The semantics of regular and large object caches are
sufficiently different to warrant a clean separation. Large
objects will very likely not be replicated across different
nodes, and I *suspect* that we will have a hard time
supporting full-blown transactions for them.</div>
</blockquote>
<blockquote type="cite">
<div bgcolor="#FFFFFF" text="#000000"> <br>
Cons:<br>
<br>
1. Increased complexity.<br>
<br>
2. Increased maintenance burden. Well, maybe. On the one
hand, there's one more kind of cache to take care of. On the
other hand the regular cache wouldn't have to pay attention
to the streaming cache's needs.<br>
<br>
3. Higher implementation effort ;-)<br>
<br>
So my proposed course of action is:<br>
<br>
1. Introduce a new interface StreamingCache that looks
exactly like the StreamingHandler above.<br>
<br>
2. Rename StreamingHandlerImpl to StreamingCacheImpl.<br>
<br>
3. Introduce a new concept, ChunkStore, a store for - you
guessed it - chunks.<br>
<br>
4. Modify StreamingCacheImpl to delegate to ChunkStore
instead of Cache.<br>
</div>
</blockquote>
Why not create a fully flagged Cache under the hood and use that
from the StreamingCacheImpl - this way you'd be able to reuse
the current implementation. Same for the configuration, the
StreamingCacheConfig would basically wrap an Configuration +
some additional fields. <br>
</div>
</blockquote>
It sounds like something I should at least try as it significantly
simplifies things. Maybe the devil's in the detail, but I will
probably start down that route.<br>
<blockquote
cite="mid:853438B0-56A1-4211-BF31-403DAA9C876F@jboss.com"
type="cite">
<div>
<blockquote type="cite">
<div bgcolor="#FFFFFF" text="#000000"> <br>
5. And now for the fun part: create
StreamingCacheConfiguration, a basic, stripped down, bare
bones, no frills variant of Configuration. Whoa, that
configuration code sure looks ... interesting. Could
probably need some pointers here.<br>
<br>
6. Pray that I won't need a custom DataContainer and so
forth.<br>
</div>
</blockquote>
you might actually, as I imagine one might want to use eviction
and passivation etc.. That would be solved with the cache
approach I described previously.<br>
</div>
</blockquote>
Well, that cache approach just grew even more compelling.<br>
<blockquote
cite="mid:853438B0-56A1-4211-BF31-403DAA9C876F@jboss.com"
type="cite">
<div>
<blockquote type="cite">
<div bgcolor="#FFFFFF" text="#000000"> <br>
What do you think? Does that make sense? Does it make
basically sense, yet you would suggest some improvements? Is
it utter nonsense?<br>
</div>
</blockquote>
<div>+1 i.e. makes total sense to me.</div>
</div>
</blockquote>
OK, then I'll give this a try. Thx.<br>
<blockquote
cite="mid:853438B0-56A1-4211-BF31-403DAA9C876F@jboss.com"
type="cite">
<div>
<blockquote type="cite">
<div bgcolor="#FFFFFF" text="#000000"> <br>
The current implementation basically works and entails
minimal changes to the existing code base. The proposed
solution is not exactly rocket science, yet considerably
more work. So I'd like to make sure that I don't head in the
wrong direction.<br>
</div>
</blockquote>
</div>
<div><br>
</div>
<div><br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
infinispan-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a>
<a class="moz-txt-link-freetext" href="https://lists.jboss.org/mailman/listinfo/infinispan-dev">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a></pre>
</blockquote>
<br>
</body>
</html>