[infinispan-dev] Strict Expiration

[Dan Berindei]

Processing expiration only on the reaper thread sounds nice, but I
have one reservation: processing 1 million entries to see that 1 of
them is expired is a lot of work, and in the general case we will not
be able to ensure an expiration precision of less than 1 minute (maybe
more, with a huge SingleFileStore attached).

What happens to users who need better precision? In particular, I know
some JCache tests were failing because HotRod was only supporting
1-second resolution instead of the 1-millisecond resolution they were
expecting.


I'm even less convinced about the need to guarantee that a clustered
expiration listener will only be triggered once, and that the entry
must be null everywhere after that listener was invoked. What's the
use case?

Note that this would make the reaper thread less efficient: with
numOwners=2 (best case), half of the entries that the reaper touches
cannot be expired, because the node isn't the primary node. And to
make matters worse, the same reaper thread would have to perform a
(synchronous?) RPC for each entry to ensure it expires everywhere.

For maxIdle I'd like to know more information about how exactly the
owners would coordinate to expire an entry. I'm pretty sure we cannot
avoid ignoring some reads (expiring an entry immediately after it was
read), and ensuring that we don't accidentally extend an entry's life
(like the current code does, when we transfer an entry to a new owner)
also sounds problematic.

I'm not saying expiring entries on each node independently is perfect,
far from it. But I wouldn't want us to provide new guarantees that
could hurt performance without a really good use case.

Cheers
Dan


On Mon, Jul 13, 2015 at 9:25 PM, Tristan Tarrant <ttarrant at redhat.com> wrote:
> After re-reading the whole original thread, I agree with the proposal
> with two caveats:
>
> - ensure that we don't break JCache compatibility
> - ensure that we document this properly
>
> Tristan
>
> On 13/07/2015 18:41, Sanne Grinovero wrote:
>> +1
>> You had me convinced at the first line, although "A lot of code can now
>> be removed and made simpler" makes it look extremely nice.
>>
>> On 13 Jul 2015 18:14, "William Burns" <mudokonman at gmail.com
>> <mailto:mudokonman at gmail.com>> wrote:
>>
>>     This is a necro of [1].
>>
>>     With Infinispan 8.0 we are adding in clustered expiration.  That
>>     includes an expiration event raised that is clustered as well.
>>     Unfortunately expiration events currently occur multiple times (if
>>     numOwners > 1) at different times across nodes in a cluster.  This
>>     makes coordinating a single cluster expiration event quite difficult.
>>
>>     To work around this I am proposing that the expiration of an event
>>     is done solely by the owner of the given key that is now expired.
>>     This would fix the issue of having multiple events and the event can
>>     be raised while holding the lock for the given key so concurrent
>>     modifications would not be an issue.
>>
>>     The problem arises when you have other nodes that have expiration
>>     set but expire at different times.  Max idle is the biggest offender
>>     with this as a read on an owner only refreshes the owners timestamp,
>>     meaning other owners would not be updated and expire preemptively.
>>     To have expiration work properly in this case you would need
>>     coordination between the owners to see if anyone has a higher
>>     value.  This requires blocking and would have to be done while
>>     accessing a key that is expired to be sure if expiration happened or
>>     not.
>>
>>     The linked dev listing proposed instead to only expire an entry by
>>     the reaper thread and not on access.  In this case a read will
>>     return a non null value until it is fully expired, increasing hit
>>     ratios possibly.
>>
>>     Their are quire a bit of real benefits for this:
>>
>>     1. Cluster cache reads would be much simpler and wouldn't have to
>>     block to verify the object exists or not since this would only be
>>     done by the reaper thread (note this would have only happened if the
>>     entry was expired locally).  An access would just return the value
>>     immediately.
>>     2. Each node only expires entries it owns in the reaper thread
>>     reducing how many entries they must check or remove.  This also
>>     provides a single point where events would be raised as we need.
>>     3. A lot of code can now be removed and made simpler as it no longer
>>     has to check for expiration.  The expiration check would only be
>>     done in 1 place, the expiration reaper thread.
>>
>>     The main issue with this proposal is as the other listing mentions
>>     is if user code expects the value to be gone after expiration for
>>     correctness.  I would say this use case is not as compelling for
>>     maxIdle, especially since we never supported it properly.  And in
>>     the case of lifespan the user could very easily store the expiration
>>     time in the object that they can check after a get as pointed out in
>>     the other thread.
>>
>>     [1]
>>     http://infinispan-developer-list.980875.n3.nabble.com/infinispan-dev-strictly-not-returning-expired-values-td3428763.html
>>
>>     _______________________________________________
>>     infinispan-dev mailing list
>>     infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>     https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>>
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>
> --
> Tristan Tarrant
> Infinispan Lead
> JBoss, a division of Red Hat
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev