<html><head><base href="x-msg://1262/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On 2 Oct 2012, at 20:23, Erik Salter wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div lang="EN-US" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Hi Mircea,<o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Awesome, thanks!<o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Initial questions:<o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><span>-<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Calculate a minimal set of nodes that can generate state: So the state provider doesn’t have to be the primary owner?</span></div></div></div></span></blockquote>AFAIK it has to, but it is not required that all nodes to be primary owners for a segment. Dan/Adrian can confirm.<br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div lang="EN-US" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><span>-<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">We are sequentially pushing each key. I presume we can leverage the xsite implementation to send (and manage) these keys in an executor (and manage the Future<?> this way)? This way, any monitoring of keys knows that they’ve been successfully pushed. Also, any thought into batching this operation?</span></div></div></div></span></blockquote>There's no 1 push per key, but they are batched up to "stateTransferCheckSize" and sent only at that point: <a href="https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/statetransfer/OutboundTransferTask.java#L217">https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/statetransfer/OutboundTransferTask.java#L217</a><br><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><span>-<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">For the tombstone functionality, I thought this might be implemented as an interceptor (with an “invoked” flag). In this case, wouldn’t we know what’s been removed and put it into the BackupReceiver.remote2localTx map accordingly?</span></div></div></div></blockquote>not sure I get your question… the tombstone refers to entries within the data container and not tx in the table. I also think that they will be implemented at data container level and not within an interceptor. <br><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; text-indent: -0.25in; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><span>-<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">What do you mean by “BackupInterceptors replicate the operations to the remote site as well”? Are you referring to the normal operation of XSite?</span></div></div></div></blockquote>yes<br><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; text-indent: -0.25in; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><span>-<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">If a key fails to replicate, do you see any danger in another pushState(site, key) command?</span></div></div></div></blockquote>a more specialised push that would only send a subset of the keys? I don't think that's a problem but it might also make sense to do a retry *during the state transfer itself* a number of times (100?).<br><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><span>-<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Error handling: We know there are some JGroups options/protocols to mitigate the remote xsite coordinator dying. In addition to that, the obvious case is an abort command. My thinking is that it would leverage some of the existing policy (like site “DOWN”) and automatically abort with an error code.</span></div></div></div></blockquote>ATM the BackupReceiver.remote2localTx is not backed up, but only leaves on one coordinator. I'm not even sure it can be backed up on another node, as it contains javax.transaction.Transaction objects. This is a tricky problem but we can always default to the optimistic approach for a start: restart the state transfer. <blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0.5in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Thanks,<o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Erik<o:p></o:p></span></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div><div style="border-right-style: none; border-bottom-style: none; border-left-style: none; border-width: initial; border-color: initial; border-top-style: solid; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding-top: 3pt; padding-right: 0in; padding-bottom: 0in; padding-left: 0in; "><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "><span class="Apple-converted-space"> </span><a href="mailto:infinispan-dev-bounces@lists.jboss.org" style="color: blue; text-decoration: underline; ">infinispan-dev-bounces@lists.jboss.org</a><span class="Apple-converted-space"> </span>[mailto:infinispan-dev-bounces@lists.jboss.org]<span class="Apple-converted-space"> </span><b>On Behalf Of<span class="Apple-converted-space"> </span></b>Mircea Markus<br><b>Sent:</b><span class="Apple-converted-space"> </span>Tuesday, October 02, 2012 12:58 PM<br><b>To:</b><span class="Apple-converted-space"> </span>infinispan<br><b>Subject:</b><span class="Apple-converted-space"> </span>[infinispan-dev] xsite state transfer design<o:p></o:p></span></div></div></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Hi,<o:p></o:p></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">After discussing with Dan and Adrian, I've updated the xsite state transfer document with a suggested design[1].<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Note that this is out of scope for 5.2. Any review is very welcomed.<o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">[1] <a href="https://community.jboss.org/wiki/DesignForCrossSiteReplication#State_Transfer_between_sites" style="color: blue; text-decoration: underline; ">https://community.jboss.org/wiki/DesignForCrossSiteReplication#State_Transfer_between_sites</a> <o:p></o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div><div><div><div><div><div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: black; ">Cheers,<o:p></o:p></span></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: black; ">-- <br>Mircea Markus<o:p></o:p></span></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: black; ">Infinispan lead (<a href="http://www.infinispan.org" style="color: blue; text-decoration: underline; ">www.infinispan.org</a>)<o:p></o:p></span></div></div><div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 13.5pt; font-family: Helvetica, sans-serif; color: black; "><o:p> </o:p></span></div></div></div></div></div></div><p class="MsoNormal" style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 12pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></p></div><div style="margin-top: 0in; margin-right: 0in; margin-left: 0in; margin-bottom: 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div></div></div>_______________________________________________<br>infinispan-dev mailing list<br><a href="mailto:infinispan-dev@lists.jboss.org" style="color: blue; text-decoration: underline; ">infinispan-dev@lists.jboss.org</a><br><a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" style="color: blue; text-decoration: underline; ">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a></div></blockquote></div><br><div apple-content-edited="true">
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Cheers,</div><div>-- <br>Mircea Markus</div><div>Infinispan lead (<a href="http://www.infinispan.org">www.infinispan.org</a>)</div><div><br></div></div></span></div></span></div></span></div></span><br class="Apple-interchange-newline"></span><br class="Apple-interchange-newline">
</div>
<br></body></html>