[infinispan-dev] Cache entry not invalidated callback?

[Galder Zamarreno]

On 11/04/2009 10:04 AM, Brian Stansberry wrote:
>
> On Nov 4, 2009, at 4:48 PM, Galder Zamarreno wrote:
>
>>
>>
>> On 11/04/2009 03:29 AM, Brian Stansberry wrote:
>>>
>>> On Nov 3, 2009, at 9:35 PM, Galder Zamarreno wrote:
>>>
>>>> Manik, I might still need the notification requested below.
>>>>
>>>> Brian, I'd like to know your thoughts on this since it might
>>>> influence
>>>> your work for
>>>> http://opensource.atlassian.com/projects/hibernate/browse/HHH-4521
>>>> (actually, I think it does! see bottom)
>>>>
>>>
>>> Yep. I'm working on HHH-4521 now and I expect this to be a problem
>>> when I run the test for it.
>>>
>>>> If we were to handle evict(key) in such way that we only set the
>>>> invalid
>>>> state for that key, the invalidation event received remotely would
>>>> need
>>>> to somehow carry the key that needs to be invalidated. Now, this
>>>> means
>>>> that without my proposed notification, you'd need to have a key
>>>> present
>>>> in the cache that is composed of address:key, since a key with
>>>> simply
>>>> address would not be sufficient to figure out which key to
>>>> invalidate.
>>>>
>>>> For evict all, this is much simpler cos key can simply be address
>>>> and
>>>> each node can initialise on startup from their local address (or
>>>> upon
>>>> viewChange). This means that when an evict all occurs, all address
>>>> instances reside in the cache, so invalidation works fine.
>>>>
>>>> However, for evict(key), it simply is not feasible to have an
>>>> address:key key for each key present in the cache in case it has
>>>> to be
>>>> evicted, and without that, the cache invalidation listener event
>>>> won't
>>>> get fired. Please also remember that the invalidation event in
>>>> Infinispan only carries the key (obviously!), so I cannot use the
>>>> value
>>>> part to add the key for example.
>>>>
>>>
>>> Yeah. For JBC I was going to remove the key from the Fqn and just use
>>> it as the value, but then realized this wouldn't work w/
>>> invalidation.
>>>
>>>> My preferred solution here would be to have a notification like the
>>>> one
>>>> suggested below. That way I don't need to have address:key for every
>>>> cluster member and key in the cache, and I can see when a particular
>>>> key
>>>> was invalidated by an address.
>>>>
>>>> Other solutions include:
>>>> - treat evict(key) as evict all: that way the key does not have to
>>>> travel to other nodes since they clear the entire cache. That would
>>>> be a
>>>> bit too extreme.
>>>>
>>>> Thinking through it, I think you'd have the same issue with JBC
>>>> based
>>>> 2nd level cache:
>>>>
>>>> Brian, When you evict a key, you do:
>>>> Fqn f = Fqn.fromRelativeElements(region, Internal.NODE, member  ==
>>>> null
>>>> ? Internal.LOCAL : member, key);
>>>> cache.put(f, ITEM, DUMMY);
>>>>
>>>> So the FQN contains the key and for it to be invalidated, you need
>>>> an
>>>> fqn with the member address and key! So, you'd have the same issue.
>>>>
>>>
>>> Yes, unless JBC emits the notification even if the node didn't exist
>>> locally. Which I doubt.
>>>
>>> TBH, this whole thing of using cache data as a mechanism to pass
>>> notifications is a hack. Much nicer IMHO would be a nice SPI / plug
>>> point to allow applications to extend the set of commands. Let an app
>>> create an ExtensionCommand; if that comes in off the channel pass it
>>> off to the handler the application registered.
>>>
>>> A worse IMHO option to the notification bus hack is to have the
>>> Hibernate-Infinispan/JBC integration create a separate JGroups
>>> channel
>>> for these messages. If a JGroups ChannelFactory + shared transport is
>>> used, that's just yuck. If ChannelFactory/shared transport aren't
>>> used, it's just unworkable.
>>
>> I agree that the best way to solve this would be for ISPN/JBC to allow
>> extension of commands but I think this requires a fair bit of thinking
>> and design to get it right. So, we'd need to design it, implement it
>> and
>> proof-test it with SPI impls for ISPN and JBC 2nd level caches.
>>
>
> Agreed. (BTW I have other use cases, i.e. session management. ;) )

I'm gonna open a JIRA for this at least in ISPN.

>
>> That's why I suggested the emit notification when a remote
>> invalidation
>> does not found the node/key in the node where is trying to do the
>> invalidation. I think this could be a good short term solution while
>> we
>> fix the real issue. I don't think it'd require a lot of code. The main
>> issue with it is that such notification is a public API, so once we
>> put
>> it out, we have to carry on maintaining it.
>>
>> WDYT?
>>
>
> Can a boolean property be added to CacheEntryInvalidatedEvent?
>
> public boolean isKeyLocallyStored()
>
> or some better name.

Rather than polluting an existing event, I'd create a new one.

However, having talked to you online earlier, we pondered about the 
urgency of fixing this for evict(key). The primary issue, and the one 
most regularly found is the handling of evictAll() but we haven't seen 
an issue with evict() yet.

So, I think we can keep the current implementation of calling remove() 
for evict(key).

>
> BTW what happens with DIST and no L1 cache? Is an invalidation message
> even sent out?

No, invalidation not is sent out.

If you start tx1 in node1 and you read a keyA, and then another tx2 in 
node2 comes and removes keyA and commits it, tx1 will fail the moment it 
tries to write keyA or the moment it tries to read it again. This is 
assuming MVCC assuming RC. With RR, tx1 might fail or not the moment it 
tries to write keyA depending on keyA write-skew options. With RR also, 
tx1 won't fail if it tries to read keyA again cos it'd read it from it's 
context and not the committed state.

>> I also agree that an extra JGroups channel wouldn't be the best thing
>> here. It looks like an overkill to me.
>>
>>
>> --
>> Galder Zamarreño
>> Sr. Software Engineer
>> Infinispan, JBoss Cache
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev

-- 
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache