[infinispan-dev] Clustered Listener

[William Burns]

On Tue, May 13, 2014 at 9:10 AM, Pierre Sutra <pierre.sutra at unine.ch> wrote:
> Hello,
>
> As part of the LEADS project, we have been using recently the clustered
> listeners API. In our use case, the application is employing a few
> thousands listeners, constantly installing and un-installing them.

Are you talking about non clustered listeners?  It seems unlikely you
would need so many cluster listeners.  Cluster listeners should allow
you to only install a small amount of them, usually you would have
only additional ones if you have a Filter applied limiting what
key/values are returned.

> The
> overall picture is that things work smoothly up to a few hundreds
> listeners, but above the cost is high due to the full replication
> schema. To sidestep this issue, we have added a mechanism that allows
> listening only to a single key.

Is the KeyFilter or KeyValueFilter not sufficient for this?

    void addListener(Object listener, KeyFilter<? super K> filter);

    <C> void addListener(Object listener, KeyValueFilter<? super K, ?
super V> filter, Converter<? super K, ? super V, C> converter);

Also to note if you are doing any kind of translation of the value to
another value it is recommended to do that via the supplied Converter.
 This can give good performance as the conversion is done on the
target node and not all in 1 node and also you can reduce the payload
if the resultant value has a serialized form that is smaller than the
original value.

> In such a case, the listener is solely
> installed at the key owners. This greatly helps the scalability of the
> mechanism at the cost of fault-tolerance since, in the current state of
> the implementation, listeners are not forwarded to new data owners.
> Since as a next step [1] it is planned to handle topology change, do you
> plan also to support key (or key range) specific listener ?

These should be covered with the 2 overloads as I mentioned above.
This should be the most performant way as the filter is replicated to
the node upon installation so a 1 time cost.  But if a key/value pair
doesn't pass the filter the event is not sent to the node where the
listener is installed.

> Besides,
> regarding this last point and the current state of the implementation, I
> would have like to know what is the purpose of the re-installation of
> the cluster listener in case of a view change in the addedListener()
> method of the CacheNotifierImpl class.

This isn't a re-installation.  This is used to propgate the
RemoteClusterListener to the other nodes, so that when a new event is
generated it can see that and subsequently send it back to the node
where the listener is installed.  There is also a second check in
there in case if a new node joins in the middle.

> Many thanks in advance.

No problem, glad you guys are testing out this feature already :)

>
> Best,
> Pierre Sutra
>
> [1]
> https://github.com/infinispan/infinispan/wiki/Clustered-listeners#handling-topology-changes
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev