<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<meta name=Generator content="Microsoft Word 14 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--><div class=WordSection1><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><span style='font-family:Wingdings'><span style='mso-list:Ignore'>Ø<span style='font:7.0pt "Times New Roman"'> </span></span></span>When you ask for 100 GB of mapped memory, on Linux doesn't need to allocate any pages to it, you just change the virtual memory table of your process. (This must in turn use some memory but the overhead is low)<o:p></o:p></p><p class=MsoListParagraph><span style='font-family:"Calibri","sans-serif"'><o:p> </o:p></span></p><p class=MsoListParagraph style='margin-left:0in'><span style='font-family:"Calibri","sans-serif";color:#1F497D'>Got it! Thanks Peter … it’s coming back to me (been a while since I last studied SystemV process subsystem internals).<o:p></o:p></span></p><p class=MsoNormal><span style='font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Peter Lawrey [mailto:<a href="/user/SendEmail.jtp?type=node&node=4028983&i=0" target="_top" rel="nofollow" link="external">[hidden email]</a>] <br><b>Sent:</b> Saturday, March 8, 2014 8:03 PM<br><b>To:</b> <a href="/user/SendEmail.jtp?type=node&node=4028983&i=1" target="_top" rel="nofollow" link="external">[hidden email]</a><br><b>Cc:</b> Ben Cotton; Justin P Dildy; Dmitry Gordeev<br><b>Subject:</b> Re: [infinispan-dev] Infinispan embedded off-heap cache<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>When you ask for 100 GB of mapped memory, on Linux doesn't need to allocate any pages to it, you just change the virtual memory table of your process. (This must in turn use some memory but the overhead is low)<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>e.g. even in plain NIO, you can create a 1 GB mapping to a new file. At this point it doesn't need to allocate any memory, nor disk space as you haven't used any of it yet. You write to every 1 MB of this 1 GB file. i.e. 1024 times. What happens? well it need to allocate at least one page i.e. 4 KB, for each write (even if just a byte). At this point the file will be 4KB * 1024, i.e. only 4 MB of memory and disk space is used at this point. If you plan for this, you can optimise the structure to only use the pages needed as much as possible.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>In the example I gave, each entry has up to 64 KB. This means each entry uses 64 KB of *address space* not memory, not disk space. So if you write only 1KB, it has to use a page, 4 KB but not the whole 64 KB. i.e. once you make the maximum size well over 4 KB, you may as well go for broke because it won't make much difference to the memory or disk space used.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>BTW, This is how Linux/UNIX behaves, Windows is lame this way and over committing memory doesn't work so well.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p><div><p class=MsoNormal>On 9 March 2014 11:52, <<a href="/user/SendEmail.jtp?type=node&node=4028983&i=2" target="_top" rel="nofollow" link="external">[hidden email]</a>> wrote:<o:p></o:p></p><div><div><p><span style='font-family:Wingdings;color:#1F497D'>Ø</span><span style='font-size:7.0pt;color:#1F497D'> </span>How you might wonder?<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:"Calibri","sans-serif";color:#1F497D'>You got that right! Let me study this … Truth is my first, second nor third reads set off the “light bulb” in my head.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p><span style='font-family:Wingdings;color:#1F497D'>Ø</span><span style='font-size:7.0pt;color:#1F497D'> </span>This works because even though I only have 7.7 GB after the OS, I can create a SHM of 137 GB because this only uses 21 MB of actual disk space/memory.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:"Calibri","sans-serif";color:#1F497D'>Still wondering ‘How?’, Peter. LOL. I’m mean I’m sure it is true …. But I have no insight to how it is true.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:"Calibri","sans-serif";color:#1F497D'>Again, Let me study the code for this one.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Peter Lawrey [mailto:<a href="/user/SendEmail.jtp?type=node&node=4028983&i=3" target="_top" rel="nofollow" link="external">[hidden email]</a>] <br><b>Sent:</b> Saturday, March 8, 2014 7:41 PM<br><b>To:</b> <a href="/user/SendEmail.jtp?type=node&node=4028983&i=4" target="_top" rel="nofollow" link="external">[hidden email]</a><br><b>Cc:</b> Ben Cotton; Justin P Dildy; Dmitry Gordeev; <a href="/user/SendEmail.jtp?type=node&node=4028983&i=5" target="_top" rel="nofollow" link="external">[hidden email]</a><br><b>Subject:</b> Re: [infinispan-dev] Infinispan embedded off-heap cache</span><o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>You might find this example interesting.<o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>While SHM is not expandable, this is not as much of a problem as it might seem. SHM uses virtual memory and leave the OS to map it to real memory as required. This means you can over allocate extents with a very low cost on Linux.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><a href="https://github.com/OpenHFT/HugeCollections/blob/master/collections/src/test/java/net/openhft/collections/OSResizesMain.java" target="_blank" rel="nofollow" link="external">https://github.com/OpenHFT/HugeCollections/blob/master/collections/src/test/java/net/openhft/collections/OSResizesMain.java</a><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>In the example above I create extents for an SHM which much larger than main memory and it takes a fraction of a second to do this. How you might wonder? It prints<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><i><span style='font-size:9.0pt;font-family:Consolas;color:#999988'>System memory= 7.7 GB, Size of map is 137.5 GB, disk used= 21MB</span></i><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><br>This works because even though I only have 7.7 GB after the OS, I can create a SHM of 137 GB because this only uses 21 MB of actual disk space/memory.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>You can freely over allocate the size on the basis that the system only uses the resources it needs.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On 8 March 2014 04:06, Ben Cotton <<a href="/user/SendEmail.jtp?type=node&node=4028983&i=6" target="_top" rel="nofollow" link="external">[hidden email]</a>> wrote:<o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><br><span style='color:#000099'>Thanks Peter. The plan is for Dmitry and I to at first extend VanillaSharedHashMap and groom it into an ISPN7 join via their DataContainer API bridge.<br><br>That ExtendedVSHM will be morphed into a fully inter-operable JCACHE operand will<br><br>- initially be brokered by the ISPN 7 config (JSR107 <----> VSHMExtendedDataContainer <----> VSHM)<br>- eventually, possibly, be rendered with ExtendedVSHM directly implementing javax.cache.Cache (in addititon to DataContainer)</span><o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On 03/07/2014 11:43 AM, Peter Lawrey wrote:<o:p></o:p></p></div><blockquote style='border-left:2px solid #CCCCCC;padding:0 1em' style='margin-top:5.0pt;margin-bottom:5.0pt'><p>In the medium term I would see SHM supporting a DataContainer. If a Cache were supported I would do it as a layered class so those who don't need the functionality of a Cache don't incur an overhead.<o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On 8 Mar 2014 03:35, "Ben Cotton" <<a href="/user/SendEmail.jtp?type=node&node=4028983&i=7" target="_top" rel="nofollow" link="external">[hidden email]</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:#000099'>Thank you for this insight Mircea ...<br><br>Ultimately ... I want the OpenHFT SHM off-heap operand to behave *exactly* like a JCACHE ... Amenable to being soundly/completely operated upon by any/all parts of ISPN7's Impl of the JSR-107 API .</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='color:#000099'>Musing openly: Won't that (eventually) necessitate me physically implementing javax.cache.Cache ?<br></span><i><br>> Another way to do it is to have CacheImpl implement the DataContainer only, and then configure Infinispan's JCache implementation to use that custom DataContainer. <br><br></i><span style='color:#000099'>I see what you mean. OK, for sure, this sounds much simpler than what I have put on my initial TODO list.<br><br>Question: Will doing it this way in any manner suggest that my JSR-107 specific operators are being transitively "brokered" by the ISPN config onto my OpenHFT SHM operand? If possible, I want everything to be direct -- no API bridge.<br><br>Thanks again, Mircea.<br><br>-Ben and Dmitry</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><br>Sent from my iPhone<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On 03/07/2014 09:45 AM, Mircea Markus-2 [via Infinispan Developer List] wrote:<o:p></o:p></p></div><blockquote style='border-left:2px solid #CCCCCC;padding:0 1em' style='margin-top:5.0pt;margin-bottom:5.0pt'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi Ben, <br><br>In the diagram provided, the CacheImpl (your class) extends both from javax.cache.Cache and org.infinispan.container.DataContainer. <br>The Cache and DataContainer interfaces are quite different and I anticipate an single class implementing both to be hard to follow and potentially not very efficient. <br>Another way to do it is to have CacheImpl implement the DataContainer only, and then configure Infinispan's JCache implementation to use that custom DataContainer. <br><br>On Mar 3, 2014, at 3:46 PM, cotton-ben <<a href="http://user/SendEmail.jtp?type=node&node=4028967&i=0" target="_blank" rel="nofollow" link="external">[hidden email]</a>> wrote: <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><div class='shrinkable-quote'><br>> Quick Update: <br>> <br>> It is my understandng that Peter Lawrey will make available an OpenHFT HC <br>> Alpha Release in Maven Central next weekend. At that time, Dmitry Gordeev <br>> and I will take the OpenHFT dependency tag and proceed to build a branch of <br>> Red Hat's ISPN 7 that will treat net.openhft.collections.SharedHashMap as a <br>> Red Hat Infinispan 7 default impl of a fully JSR-107 interoperable off-heap <br>> javax.cache.Cache ... <br>> <br>> A diagram of this build effort can be found here: <br>> <a href="https://raw.github.com/Cotton-Ben/OpenHFT/master/doc/AdaptingOpenHFT-SHM-as-JCACHE-Impl.jpg" target="_blank" rel="nofollow" link="external">https://raw.github.com/Cotton-Ben/OpenHFT/master/doc/AdaptingOpenHFT-SHM-as-JCACHE-Impl.jpg</a><br>> ... <br>> <br>> The Red Hat view of his effort will be tracked here: <br>> <a href="https://issues.jboss.org/browse/ISPN-871" target="_blank" rel="nofollow" link="external">https://issues.jboss.org/browse/ISPN-871</a> ... <br>> <br>> The code that defines the Impl will be here <br>> <a href="https://github.com/Cotton-Ben/infinispan" target="_blank" rel="nofollow" link="external">https://github.com/Cotton-Ben/infinispan</a> ... <br>> <br>> <br>> <br>> <br>> <br>> -- <br>> View this message in context: <a href="http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028931.html" target="_blank" rel="nofollow" link="external">http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028931.html</a><br>> Sent from the Infinispan Developer List mailing list archive at Nabble.com. <br>> _______________________________________________ <br>> infinispan-dev mailing list <br>> <a href="http://user/SendEmail.jtp?type=node&node=4028967&i=1" target="_blank" rel="nofollow" link="external">[hidden email]</a> <br>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank" rel="nofollow" link="external">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'></div>Cheers, <br>-- <br>Mircea Markus <br>Infinispan lead (<a href="http://www.infinispan.org" target="_blank" rel="nofollow" link="external">www.infinispan.org</a>) <br><br><br><br><br><br>_______________________________________________ <br>infinispan-dev mailing list <br><a href="http://user/SendEmail.jtp?type=node&node=4028967&i=2" target="_blank" rel="nofollow" link="external">[hidden email]</a> <br><a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank" rel="nofollow" link="external">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><o:p></o:p></p><div class=MsoNormal align=center style='text-align:center'><hr size=1 width="100%" noshade style='color:#CCCCCC' align=center></div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span style='font-size:9.0pt;font-family:"Tahoma","sans-serif";color:#444444'>If you reply to this email, your message will be added to the discussion below:</span></b><o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:9.0pt;font-family:"Tahoma","sans-serif";color:#444444'><a href="http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028967.html" target="_blank" rel="nofollow" link="external">http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028967.html</a> </span><o:p></o:p></p></div><div style='margin-top:4.8pt'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;line-height:18.0pt'><span style='font-size:8.5pt;font-family:"Tahoma","sans-serif";color:#666666'>To start a new topic under Infinispan Developer List, email <a href="/user/SendEmail.jtp?type=node&node=4028983&i=8" target="_top" rel="nofollow" link="external">[hidden email]</a> <br>To unsubscribe from [infinispan-dev] Infinispan embedded off-heap cache, <a href="" target="_blank" rel="nofollow" link="external">click here</a>.<br><a href="http://infinispan-developer-list.980875.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml" target="_blank" rel="nofollow" link="external"><span style='font-size:7.0pt;font-family:"Times New Roman","serif"'>NAML</span></a> </span><o:p></o:p></p></div></blockquote><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div></div></blockquote><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div></div></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div></div></div></div></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></div>
<br/><hr align="left" width="300" />
View this message in context: <a href="http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-Infinispan-embedded-off-heap-cache-tp4026102p4028983.html">RE: [infinispan-dev] Infinispan embedded off-heap cache</a><br/>
Sent from the <a href="http://infinispan-developer-list.980875.n3.nabble.com/">Infinispan Developer List mailing list archive</a> at Nabble.com.<br/>