[infinispan-dev] Design of Remote Hot Rod events

[Galder Zamarreño]

On Nov 19, 2013, at 1:17 PM, Pierre Sutra <pierre.sutra at unine.ch> wrote:

> Dear Galder,
> 
> I have read with great interest the design document your wrote for Hot
> Rod remote eventing [1].

I guess you mean [2] :)

> In the perspective of the LEADS project, that aims at building a
> continuous query/streaming engine on top of Infinispan, this new feature
> seem very promising. However, it seems that in the current design their
> is no mean to ensure dependability, i.e., the property that once
> registered for a key k, a client will never miss an event regarding k,
> even if the primary server in charge of k fails. How difficult do you
> think it would be to ensure this ?

The initial design has focused on making sure the last available event is delivered to the client, as opposed to all events for a particular key K. It seems like, as mentioned by Emmanuel, to build a CQ engine, you'd need to receive all events for a particular key, so we'll quite likely incorporate both options, depending on your use case.

> Besides, following Emmanuel's opinion, I believe that it would be useful
> to notify a client on a new cache insertion and to allow it to set-up
> filters on the servers (this last point is part of your clustered
> listeners design document [2] but I do not know if you plan to
> incorporate it to Hot-Rod).

Clustered listeners is a different topic since it's mostly focused around embedded listeners, or listeners that run in the same VM as Infinispan. 

In remote Hot Rod events we are focusing on cache events being fired back to Hot Rod clients. There is certainly a lot of value in having server side filters defined which control which events are fired back to Hot Rod clients. I do expect to incorporate this to the design once this round of feedback has completed.

Side note on clustered listeners vs remote hot rod events:
There are some sinergies between remote Hot Rod events and clustere listeners, in that the latter adds cluster wide state of listeners. Remote events might piggy back on this state in order to maintain information about the remote listeners on the server side. This would avoid the need for clients to register listeners when a new node joins.

Cheers,

> 
> Thank you in advance for your answers,
> Best,
> 
> Pierre
> 
> [1] https://github.com/infinispan/infinispan/wiki/Clustered-listeners
> [2] https://github.com/infinispan/infinispan/wiki/Remote-Hot-Rod-Events
> 
> 
> Le 19. 11. 13 09:48, Emmanuel Bernard a écrit :
>> Hey there,
>> 
>> Here are a few comments based on a quick reading.
>> I might have totally misread or misinterpreted what was exposed, feel
>> free to correct me.
>> 
>> ## General
>> 
>> I think you are restricting the design to listeners:
>> 
>> * that only listen to raw entry changes
>> * whose processing is remote
>> * with no way to filter out the event from the server
>> 
>> Is that correct? I can see that it does address the remote L1 use case
>> but I feel like it will close the doors to many more use cases. An
>> interesting example being continuous query.
>> 
>> In that use case the listener code runs a filtering logic server side
>> and only send keys that are impacted by the query plus some flag
>> defining whether it's added to changed or removed from the corpus.
>> The key is filtering event before sending it to the client.
>> 
>> I wish the design document was showing how we can achieve a general
>> purpose remote listener approach but have a step 1 that is only
>> targeting a restricted set of listeners if you feel that it's too much
>> to chew. I don't want us to be trapped in a situation where backward
>> compatibility prevent us from adding use cases.
>> 
>> ## Specific questions
>> 
>> When the topology changes, it is the responsibility of the client to add
>> the listener to the new servers that show up. Correct? The API is a
>> global addRemoteListener but I imagine the client implementation will
>> have to transparently deal with that.
>> I wonder if a server approach is not more convinient. At least it does
>> not put the burden and bugs in several implementations and several
>> languages.
>> 
>> You never send code at the moment. Only one kind of listener is
>> available and listeners to all entry change and deletion. Correct?
>> 
>> Why not have the ability to listen to new entry events? That would limit
>> generic listeners as it is.
>> 
>> Do you have plans to make the ACK optional depending on the listener
>> requirement? Looks like an expensive process.
>> 
>> "Only the latest event is tracked for ACK for a given key"
>> It seems it's fine for L1 but would be a problem for many more generic
>> listeners.
>> 
>> Emmanuel
>> 
>> 
>> On Tue 2013-11-12 16:17, Galder Zamarreño wrote:
>>> Hi all,
>>> 
>>> Re: https://github.com/infinispan/infinispan/wiki/Remote-Hot-Rod-Events
>>> 
>>> I've just finished writing up the Hot Rod remote events design document. Amongst many other use cases, this will enable near caching use cases with the help of Hot Rod client callbacks.
>>> 
>>> Cheers,
>>> --
>>> Galder Zamarreño
>>> galder at redhat.com
>>> twitter.com/galderz
>>> 
>>> Project Lead, Escalante
>>> http://escalante.io
>>> 
>>> Engineer, Infinispan
>>> http://infinispan.org
>>> 
>>> 
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


--
Galder Zamarreño
galder at redhat.com
twitter.com/galderz

Project Lead, Escalante
http://escalante.io

Engineer, Infinispan
http://infinispan.org