<meta http-equiv=Content-Type content="text/html; charset=iso-8859-1"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--><div class=WordSection1><p class=MsoNormal>Ah, I think I see it. <br><br>In my old System V days, I think we used to call this "your process is eating up all my 'SWAP' ..." ... which of course was *precious* and the entire basis for demand-paged Virtual Memory process internals.<br><br>So when you say my SHM can grow to 137GB virtual capacity (even on a 7.7GB physical capacity) it is because you are consuming ‘SWAP’ … correct?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I see I’ve got 2 new from you in my Inbox. Probably the answers.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>(Even if I’ve in fact solved this puzzle, I think I should spin again! Or buy a vowel?) :-)<o:p></o:p></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Ben Cotton [mailto:<a href="/user/SendEmail.jtp?type=node&node=4028982&i=0" target="_top" rel="nofollow" link="external">[hidden email]</a>] <b>On Behalf Of </b><a href="/user/SendEmail.jtp?type=node&node=4028982&i=1" target="_top" rel="nofollow" link="external">[hidden email]</a><br><b>Sent:</b> Saturday, March 8, 2014 7:53 PM<br><b>To:</b> 'Peter Lawrey'<br><b>Cc:</b> 'Ben Cotton'; 'Justin P Dildy'; 'Dmitry Gordeev'; <a href="/user/SendEmail.jtp?type=node&node=4028982&i=2" target="_top" rel="nofollow" link="external">[hidden email]</a>; <a href="/user/SendEmail.jtp?type=node&node=4028982&i=3" target="_top" rel="nofollow" link="external">[hidden email]</a><br><b>Subject:</b> RE: [infinispan-dev] Infinispan embedded off-heap cache<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo2'><span style='font-family:Wingdings;color:#1F497D'><span style='mso-list:Ignore'>Ø<span style='font:7.0pt "Times New Roman"'> </span></span></span>How you might wonder?<span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p></o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'>You got that right! Let me study this … Truth is my first, second nor third reads set off the “light bulb” in my head.<o:p></o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo2'><span style='font-family:Wingdings;color:#1F497D'><span style='mso-list:Ignore'>Ø<span style='font:7.0pt "Times New Roman"'> </span></span></span>This works because even though I only have 7.7 GB after the OS, I can create a SHM of 137 GB because this only uses 21 MB of actual disk space/memory.<span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p></o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'>Still wondering ‘How?’, Peter. LOL. I’m mean I’m sure it is true …. But I have no insight to how it is true.<o:p></o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'>Again, Let me study the code for this one.<o:p></o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Peter Lawrey [<a href="/user/SendEmail.jtp?type=node&node=4028982&i=4" target="_top" rel="nofollow" link="external">[hidden email]</a>] <br><b>Sent:</b> Saturday, March 8, 2014 7:41 PM<br><b>To:</b> <a href="/user/SendEmail.jtp?type=node&node=4028982&i=5" target="_top" rel="nofollow" link="external">[hidden email]</a><br><b>Cc:</b> Ben Cotton; Justin P Dildy; Dmitry Gordeev; <a href="/user/SendEmail.jtp?type=node&node=4028982&i=6" target="_top" rel="nofollow" link="external">[hidden email]</a><br><b>Subject:</b> Re: [infinispan-dev] Infinispan embedded off-heap cache<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>You might find this example interesting.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>While SHM is not expandable, this is not as much of a problem as it might seem. SHM uses virtual memory and leave the OS to map it to real memory as required. This means you can over allocate extents with a very low cost on Linux.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><a href="https://github.com/OpenHFT/HugeCollections/blob/master/collections/src/test/java/net/openhft/collections/OSResizesMain.java" target="_top" rel="nofollow" link="external">https://github.com/OpenHFT/HugeCollections/blob/master/collections/src/test/java/net/openhft/collections/OSResizesMain.java</a><o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>In the example above I create extents for an SHM which much larger than main memory and it takes a fraction of a second to do this. How you might wonder? It prints<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><i><span style='font-size:9.0pt;font-family:Consolas;color:#999988'>System memory= 7.7 GB, Size of map is 137.5 GB, disk used= 21MB</span></i><o:p></o:p></p></div><div><p class=MsoNormal><br>This works because even though I only have 7.7 GB after the OS, I can create a SHM of 137 GB because this only uses 21 MB of actual disk space/memory.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>You can freely over allocate the size on the basis that the system only uses the resources it needs.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On 8 March 2014 04:06, Ben Cotton <<a href="/user/SendEmail.jtp?type=node&node=4028982&i=7" target="_top" rel="nofollow" link="external">[hidden email]</a>> wrote:<o:p></o:p></p><div><p class=MsoNormal><br><span style='color:#000099'>Thanks Peter. The plan is for Dmitry and I to at first extend VanillaSharedHashMap and groom it into an ISPN7 join via their DataContainer API bridge.<br><br>That ExtendedVSHM will be morphed into a fully inter-operable JCACHE operand will<br><br>- initially be brokered by the ISPN 7 config (JSR107 <----> VSHMExtendedDataContainer <----> VSHM)<br>- eventually, possibly, be rendered with ExtendedVSHM directly implementing javax.cache.Cache (in addititon to DataContainer)</span><o:p></o:p></p><div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On 03/07/2014 11:43 AM, Peter Lawrey wrote:<o:p></o:p></p></div><blockquote style='border-left:2px solid #CCCCCC;padding:0 1em' style='margin-top:5.0pt;margin-bottom:5.0pt'><p>In the medium term I would see SHM supporting a DataContainer. If a Cache were supported I would do it as a layered class so those who don't need the functionality of a Cache don't incur an overhead.<o:p></o:p></p><div><p class=MsoNormal>On 8 Mar 2014 03:35, "Ben Cotton" <<a href="/user/SendEmail.jtp?type=node&node=4028982&i=8" target="_top" rel="nofollow" link="external">[hidden email]</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal><span style='color:#000099'>Thank you for this insight Mircea ...<br><br>Ultimately ... I want the OpenHFT SHM off-heap operand to behave *exactly* like a JCACHE ... Amenable to being soundly/completely operated upon by any/all parts of ISPN7's Impl of the JSR-107 API .</span><o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><span style='color:#000099'>Musing openly: Won't that (eventually) necessitate me physically implementing javax.cache.Cache ?<br></span><i><span style='color:black'><br>> Another way to do it is to have CacheImpl implement the DataContainer only, and then configure Infinispan's JCache implementation to use that custom DataContainer. <br><br></span></i><span style='color:#000099'>I see what you mean. OK, for sure, this sounds much simpler than what I have put on my initial TODO list.<br><br>Question: Will doing it this way in any manner suggest that my JSR-107 specific operators are being transitively "brokered" by the ISPN config onto my OpenHFT SHM operand? If possible, I want everything to be direct -- no API bridge.<br><br>Thanks again, Mircea.<br><br>-Ben and Dmitry</span><o:p></o:p></p></div><div><p class=MsoNormal><br>Sent from my iPhone<o:p></o:p></p></div><div><p class=MsoNormal>On 03/07/2014 09:45 AM, Mircea Markus-2 [via Infinispan Developer List] wrote:<o:p></o:p></p></div><blockquote style='border-left:2px solid #CCCCCC;padding:0 1em' style='margin-top:5.0pt;margin-bottom:5.0pt'><p class=MsoNormal>Hi Ben, <br><br>In the diagram provided, the CacheImpl (your class) extends both from javax.cache.Cache and org.infinispan.container.DataContainer. <br>The Cache and DataContainer interfaces are quite different and I anticipate an single class implementing both to be hard to follow and potentially not very efficient. <br>Another way to do it is to have CacheImpl implement the DataContainer only, and then configure Infinispan's JCache implementation to use that custom DataContainer. <br><br>On Mar 3, 2014, at 3:46 PM, cotton-ben <<a href="http://user/SendEmail.jtp?type=node&node=4028967&i=0" target="_blank" rel="nofollow" link="external">[hidden email]</a>> wrote: <o:p></o:p></p><div><p class=MsoNormal><div class='shrinkable-quote'><br>> Quick Update: <br>> <br>> It is my understandng that Peter Lawrey will make available an OpenHFT HC <br>> Alpha Release in Maven Central next weekend. At that time, Dmitry Gordeev <br>> and I will take the OpenHFT dependency tag and proceed to build a branch of <br>> Red Hat's ISPN 7 that will treat net.openhft.collections.SharedHashMap as a <br>> Red Hat Infinispan 7 default impl of a fully JSR-107 interoperable off-heap <br>> javax.cache.Cache ... <br>> <br>> A diagram of this build effort can be found here: <br>> <a href="https://raw.github.com/Cotton-Ben/OpenHFT/master/doc/AdaptingOpenHFT-SHM-as-JCACHE-Impl.jpg" target="_blank" rel="nofollow" link="external">https://raw.github.com/Cotton-Ben/OpenHFT/master/doc/AdaptingOpenHFT-SHM-as-JCACHE-Impl.jpg</a><br>> ... <br>> <br>> The Red Hat view of his effort will be tracked here: <br>> <a href="https://issues.jboss.org/browse/ISPN-871" target="_blank" rel="nofollow" link="external">https://issues.jboss.org/browse/ISPN-871</a> ... <br>> <br>> The code that defines the Impl will be here <br>> <a href="https://github.com/Cotton-Ben/infinispan" target="_blank" rel="nofollow" link="external">https://github.com/Cotton-Ben/infinispan</a> ... <br>> <br>> <br>> <br>> <br>> <br>> -- <br>> View this message in context: <a href="http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028931.html" target="_blank" rel="nofollow" link="external">http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028931.html</a><br>> Sent from the Infinispan Developer List mailing list archive at Nabble.com. <br>> _______________________________________________ <br>> infinispan-dev mailing list <br>> <a href="http://user/SendEmail.jtp?type=node&node=4028967&i=1" target="_blank" rel="nofollow" link="external">[hidden email]</a> <br>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank" rel="nofollow" link="external">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><o:p></o:p></p></div><p class=MsoNormal style='margin-bottom:12.0pt'></div>Cheers, <br>-- <br>Mircea Markus <br>Infinispan lead (<a href="http://www.infinispan.org" target="_blank" rel="nofollow" link="external">www.infinispan.org</a>) <br><br><br><br><br><br>_______________________________________________ <br>infinispan-dev mailing list <br><a href="http://user/SendEmail.jtp?type=node&node=4028967&i=2" target="_blank" rel="nofollow" link="external">[hidden email]</a> <br><a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank" rel="nofollow" link="external">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><o:p></o:p></p><div class=MsoNormal align=center style='text-align:center'><hr size=1 width="100%" noshade style='color:#CCCCCC' align=center></div><div><div><p class=MsoNormal><b><span style='font-size:9.0pt;font-family:"Tahoma","sans-serif";color:#444444'>If you reply to this email, your message will be added to the discussion below:<o:p></o:p></span></b></p></div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Tahoma","sans-serif";color:#444444'><a href="http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028967.html" target="_blank" rel="nofollow" link="external">http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028967.html</a> <o:p></o:p></span></p></div><div style='margin-top:4.8pt'><p class=MsoNormal style='line-height:18.0pt'><span style='font-size:8.5pt;font-family:"Tahoma","sans-serif";color:#666666'>To start a new topic under Infinispan Developer List, email <a href="/user/SendEmail.jtp?type=node&node=4028982&i=9" target="_top" rel="nofollow" link="external">[hidden email]</a> <br>To unsubscribe from [infinispan-dev] Infinispan embedded off-heap cache, <a href="" target="_blank" rel="nofollow" link="external">click here</a>.<br><a href="http://infinispan-developer-list.980875.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml" target="_blank" rel="nofollow" link="external"><span style='font-size:7.0pt;font-family:"Times New Roman","serif"'>NAML</span></a> <o:p></o:p></span></p></div></blockquote><p class=MsoNormal><o:p> </o:p></p></div></div></blockquote><p class=MsoNormal><o:p> </o:p></p></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></div></div>
<br/><hr align="left" width="300" />
View this message in context: <a href="http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028982.html">RE: [infinispan-dev] Infinispan embedded off-heap cache</a><br/>
Sent from the <a href="http://infinispan-developer-list.980875.n3.nabble.com/">Infinispan Developer List mailing list archive</a> at Nabble.com.<br/>