[infinispan-issues] [JBoss JIRA] (ISPN-11176) XSite Max Idle

[Dan Berindei (Jira)]

    [ https://issues.redhat.com/browse/ISPN-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361751#comment-14361751 ] 

Dan Berindei commented on ISPN-11176:
-------------------------------------

{quote}I have changed gears so that the system clock time does not really matter as everything should just be a replicated touch. There is no reason I can think of currently as to why we need an offset value. This makes time a lot easier as we don't have to worry about clock skews.
{quote}
(y)
{quote}Even if the remote site node that sends the async touch crashes at worst it means we could have another xsite max idle check on the remote site asking the originator site. Consistency should be fine, just possibly increased latency for a given request.
{quote}
(y)
{quote}I kept it as sync to be inline with clustered max idle where it is also sync. It being async without changing part of xsite max idle would cause some consistency issues. But this may need to be changed per other comments.
{quote}
Fair enough. When we already have one x-site RPC, one more in-cluster RPC is not going to add much to the latency of the operation anyway.

I was asking mostly because I'm trying to understand if we could maybe use a similar strategy to avoid the sync touch commands for in-cluster max-idle.
{quote}This is the same as the prior comment?
{quote}
Yeah, sorry. I meant to ask about the touch command in the remote site.
{quote}I think I should have been more specific as to what I meant by "an issue". In this case the backup and primary will both send an xsite touch command, instead of just 1.
{quote}
That doesn't sound like an issue to me, just a trade-off between having one extra in-cluster RPC (adding to the latency) and one parallel x-site RPC.

I was reading it as "we may have an issue [...] possibly losing the write", that sounded a lot scarier :)
{quote}Yes, getting the previous value would require for a write of any expired entry to also do a touch or get previous value from the other site. This seems like a lot of overhead and can be documented.

Also xsite can cause previous value to not be correct if the other site has a write that has not yet been replicated. So I personally feel like the expiration not returning the correct one should be okay.
{quote}
Sure, IRAC is going to allow commands to see a value that was already modified on the remote site, but that is only going to happen while IRAC hasn't replicated the update yet.

With expiration I believe it's possible for an entry to expire at time T, then a read at T+10m tries in Site 1 finds the old value hasn't expired yet because Site 2 has updated it at T+9.99m. Although 1m is probably a more realistic example, as the purge task is also going to remove expired entries entries every 1m (unless it can't keep up?)

I think it would be "more consistent" if we sent the local version in the "check last version" command to Site 2, and it would reply with "expired" so the local site would treat the entry as expired.

TBH if we sent the local version with the "check last version" command, maybe it wouldn't even be such a big overhead to send the value back in the response. But it seems pointless knowing that updates of non-expired are not replicated this way.
{quote}This one was in reference to listeners (which is all above). IRAC with conflict of writes will miss previous value for different writes as it wasn't replicated to the other site.
{quote}
I think that's fine, listeners are going to see a miss the previous value only if they weren't notified for that value, so the history they see is consistent.

All it means is that listeners can't expect the sequence of notifications to match the sequence of invoked cache operations, and because of topology retries that applies even without x-site.
{quote}Yeah, I need to think through it a bit more, but I believe we need to do the touch command only on primary and to fail the command as necessary so it isn't written to primary or backup.
{quote}
(y)
{quote}Agreed, will have to think through those as well, but just took a first stab when topology is stable at least.
{quote}
(y)

> XSite Max Idle
> --------------
>
>                 Key: ISPN-11176
>                 URL: https://issues.redhat.com/browse/ISPN-11176
>             Project: Infinispan
>          Issue Type: Enhancement
>          Components: Cross-Site Replication, Expiration
>            Reporter: Will Burns
>            Assignee: Will Burns
>            Priority: Major
>             Fix For: 12.0.0.Final
>
>
> Max idle expiration currently doesn't work with xsite. That is if an entry was written and replicated to both sites but one site never reads the value, but the other does. If they then need to read the value from the other site it will be expired (assuming the max idle time has elapsed).
> There are a few ways we can do this.
> 1. Keep access times local to every site. When a site finds an entry is expired it asks the other site(s) if it has a more recent access. If a site is known to have gone down we should touch all entries, since they may not have updated access times. Requires very little additional xsite communication.
> 2. Batch touch commands and only send every so often. Has window of loss, but should be small. Requires more site usage. Wouldn't work for really low max idle times as an entry could expire before the touch command is replicated.



--
This message was sent by Atlassian Jira
(v7.13.8#713008)